{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Time Series Forecasting with FLAML Library"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Introduction\n",
    "\n",
    "FLAML is a Python library (https://github.com/microsoft/FLAML) designed to automatically produce accurate machine learning models with low computational cost. It is fast and economical. The simple and lightweight design makes it easy to use and extend, such as adding new learners. FLAML can\n",
    "\n",
    " - serve as an economical AutoML engine,\n",
    " - be used as a fast hyperparameter tuning tool, or\n",
    " - be embedded in self-tuning software that requires low latency & resource in repetitive tuning tasks.\n",
    "\n",
    "In this notebook, we demonstrate how to use FLAML library for time series forecasting tasks: univariate time series forecasting (only time), multivariate time series forecasting (with exogneous variables) and forecasting discrete values.\n",
    "\n",
    "FLAML requires Python>=3.8. To run this notebook example, please install flaml with the [automl,ts_forecast] option:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: flaml[notebook,ts_forecast] in /home/dongjing/.local/lib/python3.8/site-packages (1.0.13)\n",
      "Requirement already satisfied: NumPy>=1.17.0rc1 in /usr/local/lib/python3.8/dist-packages (from flaml[notebook,ts_forecast]) (1.23.1)\n",
      "Requirement already satisfied: scipy>=1.4.1 in /usr/local/lib/python3.8/dist-packages (from flaml[notebook,ts_forecast]) (1.8.1)\n",
      "Requirement already satisfied: xgboost>=0.90 in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (1.7.1)\n",
      "Requirement already satisfied: scikit-learn>=0.24 in /usr/local/lib/python3.8/dist-packages (from flaml[notebook,ts_forecast]) (1.1.1)\n",
      "Requirement already satisfied: pandas>=1.1.4 in /usr/local/lib/python3.8/dist-packages (from flaml[notebook,ts_forecast]) (1.4.3)\n",
      "Requirement already satisfied: lightgbm>=2.3.1 in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (3.3.3)\n",
      "Requirement already satisfied: matplotlib; extra == \"notebook\" in /usr/local/lib/python3.8/dist-packages (from flaml[notebook,ts_forecast]) (3.5.2)\n",
      "Requirement already satisfied: rgf-python; extra == \"notebook\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (3.12.0)\n",
      "Requirement already satisfied: openml==0.10.2; extra == \"notebook\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (0.10.2)\n",
      "Requirement already satisfied: jupyter; extra == \"notebook\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (1.0.0)\n",
      "Requirement already satisfied: catboost>=0.26; extra == \"notebook\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (1.1.1)\n",
      "Requirement already satisfied: statsmodels>=0.12.2; extra == \"ts_forecast\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (0.13.5)\n",
      "Requirement already satisfied: hcrystalball==0.1.10; extra == \"ts_forecast\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (0.1.10)\n",
      "Requirement already satisfied: holidays<0.14; extra == \"ts_forecast\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (0.13)\n",
      "Requirement already satisfied: prophet>=1.0.1; extra == \"ts_forecast\" in /home/dongjing/.local/lib/python3.8/site-packages (from flaml[notebook,ts_forecast]) (1.1.1)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from scikit-learn>=0.24->flaml[notebook,ts_forecast]) (3.1.0)\n",
      "Requirement already satisfied: joblib>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from scikit-learn>=0.24->flaml[notebook,ts_forecast]) (1.1.0)\n",
      "Requirement already satisfied: pytz>=2020.1 in /home/dongjing/.local/lib/python3.8/site-packages (from pandas>=1.1.4->flaml[notebook,ts_forecast]) (2022.6)\n",
      "Requirement already satisfied: python-dateutil>=2.8.1 in /home/dongjing/.local/lib/python3.8/site-packages (from pandas>=1.1.4->flaml[notebook,ts_forecast]) (2.8.2)\n",
      "Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (from lightgbm>=2.3.1->flaml[notebook,ts_forecast]) (0.34.2)\n",
      "Requirement already satisfied: pillow>=6.2.0 in /usr/lib/python3/dist-packages (from matplotlib; extra == \"notebook\"->flaml[notebook,ts_forecast]) (7.0.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.4.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /usr/lib/python3/dist-packages (from matplotlib; extra == \"notebook\"->flaml[notebook,ts_forecast]) (20.3)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in /usr/lib/python3/dist-packages (from matplotlib; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.4.6)\n",
      "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.8/dist-packages (from matplotlib; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.11.0)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.8/dist-packages (from matplotlib; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.34.4)\n",
      "Requirement already satisfied: xmltodict in /home/dongjing/.local/lib/python3.8/site-packages (from openml==0.10.2; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.13.0)\n",
      "Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from openml==0.10.2; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.22.0)\n",
      "Requirement already satisfied: liac-arff>=2.4.0 in /home/dongjing/.local/lib/python3.8/site-packages (from openml==0.10.2; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.5.0)\n",
      "Requirement already satisfied: qtconsole in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.4.0)\n",
      "Requirement already satisfied: nbconvert in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (7.2.3)\n",
      "Requirement already satisfied: notebook in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (6.5.2)\n",
      "Requirement already satisfied: jupyter-console in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (6.4.4)\n",
      "Requirement already satisfied: ipykernel in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (6.17.0)\n",
      "Requirement already satisfied: ipywidgets in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (8.0.2)\n",
      "Requirement already satisfied: graphviz in /home/dongjing/.local/lib/python3.8/site-packages (from catboost>=0.26; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.20.1)\n",
      "Requirement already satisfied: plotly in /home/dongjing/.local/lib/python3.8/site-packages (from catboost>=0.26; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.11.0)\n",
      "Requirement already satisfied: six in /usr/lib/python3/dist-packages (from catboost>=0.26; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.14.0)\n",
      "Requirement already satisfied: patsy>=0.5.2 in /home/dongjing/.local/lib/python3.8/site-packages (from statsmodels>=0.12.2; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (0.5.3)\n",
      "Requirement already satisfied: workalendar>=10.1 in /home/dongjing/.local/lib/python3.8/site-packages (from hcrystalball==0.1.10; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (16.4.0)\n",
      "Requirement already satisfied: convertdate>=2.3.0 in /home/dongjing/.local/lib/python3.8/site-packages (from holidays<0.14; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (2.4.0)\n",
      "Requirement already satisfied: korean-lunar-calendar in /usr/local/lib/python3.8/dist-packages (from holidays<0.14; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (0.2.1)\n",
      "Requirement already satisfied: hijri-converter in /home/dongjing/.local/lib/python3.8/site-packages (from holidays<0.14; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (2.2.4)\n",
      "Requirement already satisfied: cmdstanpy>=1.0.4 in /home/dongjing/.local/lib/python3.8/site-packages (from prophet>=1.0.1; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (1.0.8)\n",
      "Requirement already satisfied: setuptools>=42 in /usr/local/lib/python3.8/dist-packages (from prophet>=1.0.1; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (59.5.0)\n",
      "Requirement already satisfied: tqdm>=4.36.1 in /usr/local/lib/python3.8/dist-packages (from prophet>=1.0.1; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (4.64.0)\n",
      "Requirement already satisfied: LunarCalendar>=0.0.9 in /home/dongjing/.local/lib/python3.8/site-packages (from prophet>=1.0.1; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (0.0.9)\n",
      "Requirement already satisfied: setuptools-git>=1.2 in /home/dongjing/.local/lib/python3.8/site-packages (from prophet>=1.0.1; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (1.2)\n",
      "Requirement already satisfied: ipython-genutils in /home/dongjing/.local/lib/python3.8/site-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.2.0)\n",
      "Requirement already satisfied: pyzmq>=17.1 in /home/dongjing/.local/lib/python3.8/site-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (24.0.1)\n",
      "Requirement already satisfied: jupyter-core in /home/dongjing/.local/lib/python3.8/site-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.11.2)\n",
      "Requirement already satisfied: pygments in /usr/local/lib/python3.8/dist-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.12.0)\n",
      "Requirement already satisfied: traitlets!=5.2.1,!=5.2.2 in /usr/local/lib/python3.8/dist-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.3.0)\n",
      "Requirement already satisfied: qtpy>=2.0.1 in /home/dongjing/.local/lib/python3.8/site-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.2.1)\n",
      "Requirement already satisfied: jupyter-client>=4.1 in /home/dongjing/.local/lib/python3.8/site-packages (from qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (7.4.4)\n",
      "Requirement already satisfied: bleach in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.0.1)\n",
      "Requirement already satisfied: defusedxml in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.7.1)\n",
      "Requirement already satisfied: jinja2>=3.0 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (3.1.2)\n",
      "Requirement already satisfied: jupyterlab-pygments in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.2.2)\n",
      "Requirement already satisfied: beautifulsoup4 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.11.1)\n",
      "Requirement already satisfied: markupsafe>=2.0 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.1.1)\n",
      "Requirement already satisfied: tinycss2 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.2.1)\n",
      "Requirement already satisfied: pandocfilters>=1.4.1 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.5.0)\n",
      "Requirement already satisfied: importlib-metadata>=3.6; python_version < \"3.10\" in /usr/local/lib/python3.8/dist-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.12.0)\n",
      "Requirement already satisfied: nbformat>=5.1 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.7.0)\n",
      "Requirement already satisfied: nbclient>=0.5.0 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.7.0)\n",
      "Requirement already satisfied: mistune<3,>=2.0.3 in /home/dongjing/.local/lib/python3.8/site-packages (from nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.0.4)\n",
      "Requirement already satisfied: Send2Trash>=1.8.0 in /home/dongjing/.local/lib/python3.8/site-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.8.0)\n",
      "Requirement already satisfied: terminado>=0.8.3 in /home/dongjing/.local/lib/python3.8/site-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.17.0)\n",
      "Requirement already satisfied: argon2-cffi in /home/dongjing/.local/lib/python3.8/site-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (21.3.0)\n",
      "Requirement already satisfied: nest-asyncio>=1.5 in /home/dongjing/.local/lib/python3.8/site-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.5.6)\n",
      "Requirement already satisfied: nbclassic>=0.4.7 in /home/dongjing/.local/lib/python3.8/site-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.4.8)\n",
      "Requirement already satisfied: prometheus-client in /usr/local/lib/python3.8/dist-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.13.1)\n",
      "Requirement already satisfied: tornado>=6.1 in /home/dongjing/.local/lib/python3.8/site-packages (from notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (6.2)\n",
      "Requirement already satisfied: ipython in /usr/local/lib/python3.8/dist-packages (from jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (8.4.0)\n",
      "Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (3.0.30)\n",
      "Requirement already satisfied: psutil in /usr/local/lib/python3.8/dist-packages (from ipykernel->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.9.1)\n",
      "Requirement already satisfied: debugpy>=1.0 in /home/dongjing/.local/lib/python3.8/site-packages (from ipykernel->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.6.3)\n",
      "Requirement already satisfied: matplotlib-inline>=0.1 in /usr/local/lib/python3.8/dist-packages (from ipykernel->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.1.3)\n",
      "Requirement already satisfied: widgetsnbextension~=4.0 in /home/dongjing/.local/lib/python3.8/site-packages (from ipywidgets->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.0.3)\n",
      "Requirement already satisfied: jupyterlab-widgets~=3.0 in /home/dongjing/.local/lib/python3.8/site-packages (from ipywidgets->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (3.0.3)\n",
      "Requirement already satisfied: tenacity>=6.2.0 in /home/dongjing/.local/lib/python3.8/site-packages (from plotly->catboost>=0.26; extra == \"notebook\"->flaml[notebook,ts_forecast]) (8.1.0)\n",
      "Requirement already satisfied: lunardate in /home/dongjing/.local/lib/python3.8/site-packages (from workalendar>=10.1->hcrystalball==0.1.10; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (0.2.0)\n",
      "Requirement already satisfied: backports.zoneinfo; python_version < \"3.9\" in /home/dongjing/.local/lib/python3.8/site-packages (from workalendar>=10.1->hcrystalball==0.1.10; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (0.2.1)\n",
      "Requirement already satisfied: pyluach in /usr/local/lib/python3.8/dist-packages (from workalendar>=10.1->hcrystalball==0.1.10; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (2.0.0)\n",
      "Requirement already satisfied: pymeeus<=1,>=0.3.13 in /home/dongjing/.local/lib/python3.8/site-packages (from convertdate>=2.3.0->holidays<0.14; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (0.5.11)\n",
      "Requirement already satisfied: ephem>=3.7.5.3 in /home/dongjing/.local/lib/python3.8/site-packages (from LunarCalendar>=0.0.9->prophet>=1.0.1; extra == \"ts_forecast\"->flaml[notebook,ts_forecast]) (4.1.3)\n",
      "Requirement already satisfied: entrypoints in /usr/lib/python3/dist-packages (from jupyter-client>=4.1->qtconsole->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.3)\n",
      "Requirement already satisfied: webencodings in /home/dongjing/.local/lib/python3.8/site-packages (from bleach->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.5.1)\n",
      "Requirement already satisfied: soupsieve>1.2 in /home/dongjing/.local/lib/python3.8/site-packages (from beautifulsoup4->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.3.2.post1)\n",
      "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata>=3.6; python_version < \"3.10\"->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (3.8.1)\n",
      "Requirement already satisfied: fastjsonschema in /home/dongjing/.local/lib/python3.8/site-packages (from nbformat>=5.1->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.16.2)\n",
      "Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.1->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.7.2)\n",
      "Requirement already satisfied: ptyprocess; os_name != \"nt\" in /home/dongjing/.local/lib/python3.8/site-packages (from terminado>=0.8.3->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.7.0)\n",
      "Requirement already satisfied: argon2-cffi-bindings in /home/dongjing/.local/lib/python3.8/site-packages (from argon2-cffi->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (21.2.0)\n",
      "Requirement already satisfied: jupyter-server>=1.8 in /home/dongjing/.local/lib/python3.8/site-packages (from nbclassic>=0.4.7->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.21.0)\n",
      "Requirement already satisfied: notebook-shim>=0.1.0 in /home/dongjing/.local/lib/python3.8/site-packages (from nbclassic>=0.4.7->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.2.2)\n",
      "Requirement already satisfied: jedi>=0.16 in /usr/local/lib/python3.8/dist-packages (from ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.18.1)\n",
      "Requirement already satisfied: stack-data in /usr/local/lib/python3.8/dist-packages (from ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.3.0)\n",
      "Requirement already satisfied: pexpect>4.3; sys_platform != \"win32\" in /usr/lib/python3/dist-packages (from ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (4.6.0)\n",
      "Requirement already satisfied: pickleshare in /usr/local/lib/python3.8/dist-packages (from ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.7.5)\n",
      "Requirement already satisfied: backcall in /usr/local/lib/python3.8/dist-packages (from ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.2.0)\n",
      "Requirement already satisfied: decorator in /usr/local/lib/python3.8/dist-packages (from ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.1.1)\n",
      "Requirement already satisfied: wcwidth in /usr/local/lib/python3.8/dist-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.2.5)\n",
      "Requirement already satisfied: importlib-resources>=1.4.0; python_version < \"3.9\" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (5.8.0)\n",
      "Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (21.4.0)\n",
      "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.18.1)\n",
      "Requirement already satisfied: cffi>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from argon2-cffi-bindings->argon2-cffi->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.15.1)\n",
      "Requirement already satisfied: websocket-client in /usr/local/lib/python3.8/dist-packages (from jupyter-server>=1.8->nbclassic>=0.4.7->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.3.3)\n",
      "Requirement already satisfied: anyio<4,>=3.1.0 in /home/dongjing/.local/lib/python3.8/site-packages (from jupyter-server>=1.8->nbclassic>=0.4.7->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (3.6.2)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.0 in /usr/local/lib/python3.8/dist-packages (from jedi>=0.16->ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.8.3)\n",
      "Requirement already satisfied: executing in /usr/local/lib/python3.8/dist-packages (from stack-data->ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.8.3)\n",
      "Requirement already satisfied: pure-eval in /usr/local/lib/python3.8/dist-packages (from stack-data->ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (0.2.2)\n",
      "Requirement already satisfied: asttokens in /usr/local/lib/python3.8/dist-packages (from stack-data->ipython->jupyter-console->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.0.5)\n",
      "Requirement already satisfied: pycparser in /usr/local/lib/python3.8/dist-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.21)\n",
      "Requirement already satisfied: idna>=2.8 in /usr/lib/python3/dist-packages (from anyio<4,>=3.1.0->jupyter-server>=1.8->nbclassic>=0.4.7->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (2.8)\n",
      "Requirement already satisfied: sniffio>=1.1 in /home/dongjing/.local/lib/python3.8/site-packages (from anyio<4,>=3.1.0->jupyter-server>=1.8->nbclassic>=0.4.7->notebook->jupyter; extra == \"notebook\"->flaml[notebook,ts_forecast]) (1.3.0)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install flaml[automl,ts_forecast] matplotlib openml\n",
    "# avoid version 1.0.2 to 1.0.5 for this notebook due to a bug for arima and sarimax's init config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Forecast Problem\n",
    "\n",
    "### Load data and preprocess\n",
    "\n",
    "Import co2 data from statsmodel. The dataset is from “Atmospheric CO2 from Continuous Air Samples at Mauna Loa Observatory, Hawaii, U.S.A.,” which collected CO2 samples from March 1958 to December 2001. The task is to predict monthly CO2 samples given only timestamps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "import statsmodels.api as sm\n",
    "data = sm.datasets.co2.load_pandas().data\n",
    "# data is given in weeks, but the task is to predict monthly, so use monthly averages instead\n",
    "data = data['co2'].resample('MS').mean()\n",
    "data = data.bfill().ffill()  # makes sure there are no missing values\n",
    "data = data.to_frame().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# split the data into a train dataframe and X_test and y_test dataframes, where the number of samples for test is equal to\n",
    "# the number of periods the user wants to predict\n",
    "num_samples = data.shape[0]\n",
    "time_horizon = 12\n",
    "split_idx = num_samples - time_horizon\n",
    "train_df = data[:split_idx]  # train_df is a dataframe with two columns: timestamp and label\n",
    "X_test = data[split_idx:]['index'].to_frame()  # X_test is a dataframe with dates for prediction\n",
    "y_test = data[split_idx:]['co2']  # y_test is a series of the values corresponding to the dates for prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABNIklEQVR4nO29eZgcZ3no+3t7Zrp7Znr26Vk0M9JotSxbtizLC4tXYmMMCQQSAgFMEhLfE+AkIZCc5CQ5uZDwJDmHmwVCuBcCBDgkHCCQELM6YGMb40WStVrbSJpVs/RsvS/T3d/9o5aunpFkSZ6WRtL7e55+VPN9VdVfldv11ruLMQZFURRFAfBd6gUoiqIoKwcVCoqiKIqLCgVFURTFRYWCoiiK4qJCQVEURXGpvtQLeDm0t7eb/v7+S70MRVGUy4pdu3ZNG2PCp5u7rIVCf38/O3fuvNTLUBRFuawQkaEzzan5SFEURXFRoaAoiqK4qFBQFEVRXFQoKIqiKC4qFBRFURQXFQqKoiiKiwoFRVEUxUWFgqIoygomEs/ymSdOsGto9qJ832WdvKYoinKl8xtf3MmekXnWttfz2Ifurvj3qaagKIpyiZlN5lgoFJeM5/JF9o9FATg1n+ZiNEVToaAoinIJKRQNd/zVj7jlo/9JNl8omxueTVIoGrb2NJHNF4mmFyq+nooJBREJishzIrJXRA6KyIft8SdFZI/9OSUi/2aPi4h8XEQGRGSfiGyv1NoURVFWChOxDMlcgfnUAk8PzJTNDUwlAHjlhjYAJmPZiq+nkppCFrjXGHMjsA14QERuN8bcYYzZZozZBvwU+Ia9/+uAjfbnYeBTFVyboijKimBkNuVu/+ehybI5Ryi8Yp0lFCZimYqvp2JCwVgk7D9r7I9rEBORRuBe4N/soTcCX7SPewZoFpHuSq1PURRlJeAIhZ7mWg6eipXNDUwlWNUUZH04BMDk5SwUAESkSkT2AFPAo8aYZz3TbwJ+aIxx7kIPMOKZH7XHFp/zYRHZKSI7I5FIZRauKIpykRiZS+MTeOX6NoZmkmVzA5EE6ztChBsCAExGMxhjSOcKpzvVslBRoWCMKdhmol7gVhG53jP9duBfLuCcnzbG7DDG7AiHT9sjQlEU5bLh6EScvtY6NnSEmEstEMtYzuSFQpETkSTrwyGCNVW01NUwEcswn1rg2v/xPb7008GKrOeiRB8ZY+aBx4AHAESkHbgV+LZntzGgz/N3rz2mKIpy2fPVnSO89m+eILNQess3xrBzaI6bV7ewpq0OgOEZy5z0zRfGSOUK3LGxHYDOxiCTsSwjc9Z8uCFYkXVWMvooLCLN9nYtcB9w2J7+BeARY4zXQPYt4CE7Cul2IGqMGa/U+hRFUS4mf/TN/RyZjPON3aV33eHZFNOJLNvXtNDbYgmFsfk0AI++OMnq1jru3dwBOEIhw8isNb+6ta4i66ykptANPCYi+4DnsXwKj9hzb2Op6eg7wAlgAPgM8N4Krk1RFOWikczmcfLOfjIw7Y7vHJwDYEd/C20hP2AlsgEcn0qwpbsREQGgyxEKtqbQ11pbkbVWrMyFMWYfcNMZ5u4+zZgB3lep9SiKolwqjkcS5IuWVDgxXXIm7xqeoyFQzaaOBhaKVkbzbDJHLl9kaDbFg1tLAZidjQGmE1kGp5M019XQEKypyFo1o1lRFKXCnLQFwR0b2xmcTlK0BcTBUzG29jbh8wmB6ipCgWpmEjmGZqxM5g0dIfccnU1BigZeGJ6nr6UypiNQoaAoilJxTk4nEYG7r+kgvVBwk9CGZpKsba9392ut9zOTzDJkO5v7PXNdjZZj+chkvGKmI9AqqYqiKBVncDrJqqZa983/1Hyaen8186kFN+oILKEwm8yV/AYtpYd/Z2Mp2qiSmoIKBUVRlApzciZFf3sd4ZCVhDYVz+Kvtgw1a9pK2kBbvZ/xaIbRuTS1NVW01vvdOa9Q6K1Q5BGo+UhRFGXZeHpgmv/z/HDZmDGGk5EEa9vr6Wi0hUIsw6BtIjqtpjCboq+11o08AktgOHg1iOVGNQVFUZRlIJcv8sv/aFXyeeuOPveBbmUp5+lvq6e1zk+1T4gksiSyeaA836A15JiP0m7egoPPJ1T7hHzRsH1NS8WuQ4WCoijKMrBvdN7djiSydNgZxyenrbqga9vr8fmE9lCAqVgWA3Q0BKjzlx7D7fUBcoUiRyZi3Nq/9MH/vd+5g6ZaP40VCkcFFQqKoijLgres9YlI0hUKB8asmp9bVjUC0NEYYCqeJZ0r0O/xJwCuD6FoWKIpAGzoaKjI2r2oT0FRFGUZ8DbAOREpJajtHZ2nPRRwQ0rDIUsoDM4kWd1W/uBvDXn8BhUMOz0bKhQURVHOg/f/827e+Y/PLhmfimWoqRIC1T5ORBLu+IGxKDf0Nrk+ho7GAMMzSabiWfoXCQWvM/l0msLFQM1HiqIo50hmocAj+6w6nSenyxPPpuJZupqC1Pur3VIWxhhG59LcsbFU5j/cECRp90NYcwbzEbBEi7hYqKagKIpyjjgF7KC8sB1YXdE6G6wuaY6mEE0vkMoV6G4q5Rh02A1zoDwcFaCtvjRXSWfy2VChoCiKco48cSxClc8yAzklrh1OTidZ1VzLunA9I3Npcvkip+Yt5/Oq5pJ/oEwotJZrCrX+Kn7vtdfwnd+6o1KX8JKo+UhRFOUceeJohNvWtjI6l2ZsriQURudSjEcz3LymhWCNj0LREElkOWULjjKh4MlMbqpbqg28754NFbyCl0Y1BUVRlEX84OAEB8aiZWOJbJ4jk3FuW9tGT3Mto3Z9IiiZlW7pb3X7KUfiWcajtlDwmI96bAHxG3esreg1XCiqKSiKongYnknx8Jd2IQIn/+L17viRiRjGwHWrGhmdS/HEsYg799zgLA2Baq7paiBv90WIxLOMzVsRSe2hksko3BDg2f/+mjIz0kpCNQVFURQPj+w/BYAxcGwy7o6/eKqUhNbVFCQSz1Kw+yLsHJzl5v4WqnyyRFPoagri80nZd3Q2BsvqGq0kVCgoiqJ4mIyWMpMHphJl26FANd1NQdrq/RQNzKdyzCVzHJ1McEt/K1CKIJq2fQqrmi5NEtqFokJBURTFw2Qs677teyOMnPLXIkJ7g/Pgz7FrqORPAPBX+2ipqyESz3JqPlPmZL4cUKGgKIriYTKeYVNniHp/VZlQGJpJurWKHG1gJpFl1/AcNVXCDb1N7r7toQCTsQwTsQyrmoNcTqhQUBRF8TAVy9LZEGRVc60bUrpQKDI6l3YzmMMNVuZxJJFleDZFb0sdwZoq9xwt9X6OTsYpFA3daj5SFEVZ+fzLc8NlNYrAKksRiWfpaHSEguVfGJlNUSgatyxFSVPIMRnN0NlYHknUWud3m+j0qPlIURRlZXNoPMYffmM/7//nF8rGI/EsuUKRrsYAbXbDG4Ah+wG/tt0qS9FUW0OVT5hJZpmIZdwKqA4t9aWktG41HymKoqxs/m3PGFDeAwFg36iVsHZdTxMtdX7mUpZQOGkXuHM0BZ9PaKv3E4lnLXNT0yKhUFcqbKeOZkVRlBXO0Qkr/2A2mXO1AbC6p/nESlBrqashlSuQzRcYmknSEKguK23dFgpwbCphaxblQsFb7fRSFba7UComFEQkKCLPicheETkoIh+2x0VEPioiR0XkkIj8lmf84yIyICL7RGR7pdamKMrVzcnpJLW2Y3hoptQQZ99YlI0dDdT5q2my3/ajqQVOzqRYY4ejOrSH/OwdmQdY4kxuto9tDF5+RSMqqSlkgXuNMTcC24AHROR24FeAPmCzMeZa4Cv2/q8DNtqfh4FPVXBtiqJc4Rhj+McnTzAVLzcRLRSKjMyledWGdqCUi2CMYd9o1A0tbbGL1c2lFsrCUR3aQwHshGY2dJTPVdsZzFs9YaqXCxUTCsbCce3X2B8D/CbwEWNM0d5vyt7njcAX7eOeAZpFpLtS61MU5crm4KkYf/7tQ3zwq3vLxp1IoldvaANww05H59LMJnPc0NcMlPwCkXi2LBzVwTElVftkSbOcm1Zb5/idn9m0rNd0MaioT0FEqkRkDzAFPGqMeRZYD/ySiOwUke+KyEZ79x5gxHP4qD22+JwP28fujEQii6cVRbmKKBYNPzg4Qb5QXDJ38JTlNHZqFjkM2uairb3NNASr3RLY++2qqDf0WG/3zbamsH8sWhaO6tBmF7nraAhQU1X+KF3TVs/gX77ezXK+nKioUDDGFIwx24Be4FYRuR4IABljzA7gM8DnzvOcnzbG7DDG7AiHwy99gKIolzXZfIFHX5w87dy39p7i4S/t4lOPH18y5zzk51K5MqFxctoJL62np7mWMTsXYe/oPDVVwubuBqDkF9gzYpWxWNxPubHW8hfctq7tgq9tJXJRoo+MMfPAY8ADWBrAN+ypbwI32NtjWL4Gh157TFGUq5g/f+QQv/HFneyxnbpenH7Jzr9e9o9ZGkLRWH4Bh5PTCRqD1bTU1RBuCDCdyFr7j0bZ3NVIoNpyQIdDAURg97D1vYtDS1+/tZs/evBa/uLNW1/2Na4kKhl9FBaRZnu7FrgPOAz8G3CPvdtdwFF7+1vAQ3YU0u1A1Biz9L+0oihXFU7fgsHp5JK5/WPzAIx4Gt6A5Uw+NB5zs4lnkll37vB4nHXhECJWroETknpyOsnGzpC7n7/aR3ejVSLbJ7hF8hya6/z8xp3ryspbXAlUUlPoBh4TkX3A81g+hUeAvwTeIiL7gb8Aft3e/zvACWAAy6z03gquTVGUy4TpuPVAP+rpbQCQyuWZjGWp91eRyhXILBTcuWOTCXL5InddY5mYZxPWg382mWP38Bx3brQij9pCAWYSWfKFIpOxzJKSFL2tlskofBq/wZVKxYJojTH7gJtOMz4PvP404wZ4X6XWoyjK5Uc8s0AyZz3sj06W1ykatH0D29e08OSxaWaSOfehfmzKEiCvXN/GPz87zLStDTx7YoaigXs2dwBWklkyV2BoNkXRLDURrW6t47mTs3RdZkXtXg5Xh+hTFOWyZNzT8GYili6bc6KIbl7TApS0AbDCSwG22eGls7bfwBlf32GZidpDljPZ6cfcvahcxbqwFXEUuEq0BNAezYqirGCcHIK17fVlD30oCYXtqy2h4PUbjMymaA/56W6qxSe4foOx+TShQLVbesKpdrrfrnm02Hz0rtvXcGg8zs9c27Hcl7ZiUaGgKMqKxSldfd2qRh59cRJjjFtqYnA6SbghQJ9t9/fWMBqZs3ocVPmEljq/az46NZ8ua3rTZmsKTiG87kVCoSFYwyfevsQKfkVz9ehEiqKsWL7w9CD9f/DtMmcxwHg0jU/g2u5Gsvkiac/84HSK/rY6t/icIxQWCkVePBVzTT9tIb+rZYxHM2V1ipztXcNzNAarCQX0PVmFgqIol5yPfucQAMcXNb0Zm0/T2Rh0w0FnPCakk3Y9osZgNTVVwowtFJ48FmEutcCD11tVclrr/a5pabGmYGUjC4WiuexKXFcKFQqKolxyinZlucPj5WGn43bj+9a6cm1gOpElEs+yqbMBEctE5GgDu4fmqfIJd2zyhJ0mcySzeWaSOXpbSpnJPp+4wkCFgoUKBUVRLinpXIG8LRQOjZfXKRqPpuluCtIaKhcKe+ws42124TlLG7DmBmeS9DTXupnJbfV+ZhI5N/LI8UE4dNq9EFZdZh3SKoUKBUVRLhpWOlI5u4bm3O1BT28DYwynopam4FQkdR78L4zMUeUTrl9lFa+zWmdaJqKhmRT9noqmbfUBoukFt3taX0u5RrDRDk91opiudlQoKIpyUfizR17kjZ/8CYViuWB4+vg01T7hFeva3Ld5sPsl54usagq6zuQ5R1MYmWdzVwO1fksbaK0PMJvMYYxhcCZZVryu1Y0wmgeWagofvP8avvX+V/Hm7b3Le8GXKSoUFEWpOEMzST771En2jUZ5+vh02dz+sSibuxu4pquBsbm0q00ctltmbupqIBQoOZOLRcO+kaibmAa2iSiZYy61QDyTLytz3W4LlBeG56mtqSprqQmW6emG3mYUCxUKiqJUnJHZkgbw+JHyPijHpxJs7Gigp7mWeDZPLJ0HSv6Fa7saERFa6y0T0cmZJPFsnhs9QqG13k88k+eYXR/Jqyl02lnKzw/O0tdaW9ZSU1mKCgVFUSrOZMxKQgvW+Mp6IiezeU5FM2zoCNFr2/qdiqcHT8XoagzSYr/ZOyYi5/j14VJF03a74c1O2z/h1RScLOV80dDXUm46UpaiQkFRlGUjmc0TTS8sGZ+0+yTf0t/K0EypzLWTl7A+HHLf6KfiGRYKRX58NMIr1pca2DhlrktRRCWHsZPH8PzgLD4pn2sPBdyeyYv9CcpSVCgoirJsvP0zz3Djh3/g5h04TMWyNASruaazgeHZlDs/MGUJhQ0dITrsB3sknuX5wVmi6QVed32Xe45Wj1AIVPsIh0r9DZxjf3p8hp6WUjgqQJVP8NlCobdFcxFeChUKiqIsC+lcwa0h9PTxmbK5yViGzsYga9rqyOaLROyqpQNTCbvxfZ1rAorEs25Z7Ovtfslg5yIkcozMpuhpKfcNdDRax2bzRbb1LQ0tzeWtdpx3btIWvi+FFvpQFGVZ2D1cyjfYOzrPq+1GNmDlDnQ3Bd0H/0wiR2djkIGpBP3t9dRU+aipgsZgNVPxLOmFAlU+cRPLwHrwx7N5jk0llvgGnGqnADvWLBUKn333DmaTOTZ1Nizb9V6pqFBQFGVZ8PoKnJLXYOUWHJqI8YGf2eQ6jedSVr7BQCTBpo7Sg7rDbn8ZSy/Q1RikylfSBjobLAExMJXgtrWtZd/try4ZPbzCyOE113a+nEu7qlChoCjKsjA2n6LaJ6wPh5jwNMf5p6cHMcZ6WDtVSGeTOXL5IkMzKbdwHUA4FCASzyICPYvs/16tofc0UURO/oE3Kkk5f1QoKIqyLIzOpeluDtLbUsspWygYY/jsUye5f0snN/U1M20XrZtLWaGlhaJhfUcpfLSjMcALw/PkC0VuX9dWdv7OxpKJ6HQO48d/726qfeomfbmoUFAUZVkYm0vT01xLV1OQXbZ/YS61QCKb57Z1bYgIzXVWx7PZZK4UeRQumY/CoQDj0TSFolmiKXR4NIXThZY22N3UlJeHilVFUV42qVyeI5Nx1rTW090UZD61QGahwKidiOYUoaup8tEYrGbOIxS8mkK4IcBCwVA0S0tZNwarabDNT9eow7hiqKagKMp58ftf30tnY5AP3n+NO/Yfe08Rz+R56y297sN+OpF1E828PoDWej+zqQWi6QV6mmup85ceQx0eE9Hifskiwg8/eBcNwRq3EJ6y/KhQUBTlnMkXinx15ygAH/iZTW5S2LHJBLU1VWxf3eJmNEfiWUZmLU2h15Nh3FLvZyaRJZZZYH1HuVM4HCqZiBabj6DchKRUBjUfKYpyzpyYLtUt2j8WdbdH59L02gllzoN92m5s01RbQ6PH3t/ZEGQiluH4VJINiyKFzqYpKBcHFQqKopwzB0+VBIG3Ic7ofMqNCGpvsEJDI/Eso3OpJZFCXU1BTkSSpBcKbFikKXTZ9Y9etaGNYI2aiC4FFTMfiUgQeAII2N/zdWPMn4rIPwF3Ac6v61eMMXvEyln/O+BBIGWP767U+hRFOX9OTpcS1JzKp2BpCjfZ5SWc7OJIPMvIXJr14fqyc3jzDRYLhcZgDT/84F2s0cJ1l4xK+hSywL3GmISI1ABPich37bnfM8Z8fdH+rwM22p/bgE/Z/yqKcpE5OZ2kLeQvM/sAjM6mWNUUZD69wGTMql8Uzywwn1pwfQD+ah8tdTVMxjOMzqW4e1G9IW++wWKhAJp8dqk5L/ORiLSIyA3nsq+xSNh/1tifpQ1aS7wR+KJ93DNAs4h0n2V/RVEqQDpX4J6PPc7DX9y5ZG5kLkVvax2djUFXUxibdyKMSmairqZaDo5FySwUlziMuzyaQuuiLmjKpeclhYKIPC4ijSLSCuwGPiMif30uJxeRKhHZA0wBjxpjnrWnPioi+0Tkb0TEeW3oAUY8h4/aY4vP+bCI7BSRnZFIZPG0oigvk6cGrHaZz5yYXTI3PJuir6WOjoYAU7amMDq7NOx0VVOQvXbF1MXawDVdDWzoCPGpd2yvyPqVl8e5aApNxpgY8GasN/nbgJ85l5MbYwrGmG1AL3CriFwP/CGwGbgFaAX+2/ks2BjzaWPMDmPMjnBYy+AqynLz/GBJGHj9BpmFApOxLKttTWHCnnMS1LyaQndzSRtYnGjWFgrwn797F6/bqoaAlci5CIVq24zzVuCRC/kSY8w88BjwgDFm3DYRZYHPA7fau40BfZ7Deu0xRVEuIt5idoOeEFRvQ5zellpOzVvlKEbn0gRrfG5BOihlI/urfG5XNOXy4FyEwkeA7wMDxpjnRWQdcOylDhKRsIg029u1wH3AYcdPYEcbvQk4YB/yLeAhsbgdiBpjxs/zehRFOQei6QXW/eG3+erOkSVzU/GMa+sf9wgIRyhs6gzR11pHvmgYj6YZnLFMSt6mN06Z63Xh+rJxZeXzkkLBGPM1Y8wNxpj32n+fMMa85RzO3Q08JiL7gOexfAqPAF8Wkf3AfqAd+HN7/+8AJ4AB4DPAe8/7ahRFOSd+cHCCooH/+b3DS+amYllu6LU6nnmFwqGJmN0lrd5tcjMym+boZJxNXeUmojs3hXn7rX188T23olxenDEkVUQ+wVmihYwxv3W2Extj9gE3nWb83jPsb4D3ne2ciqIsDz86PAVAKlcgXyhSXVV6P5yKZ7nrmjC7h+aYiFpOZGMM3z8wwa1rW/FX++izy1YcnYwzPJviF27uLTt/uCHAX7z5nAIVlRXG2fIUlsajKYpyReD0Uk7lCkzFs64PIJnNk8jm6WgI0t1U6oswMJVgcCbF/3XXesDyGYjA40cs4bKpU3MLrhTOKBSMMV/w/i0idcaY1Jn2VxRlZRHPLJBeKNDRUF5EbjaZY2w+ze3rWnnmxCyTsYwrFJw2mt1NQTqbgkzZEUZHJy1/gmNWqqny0VYfYOeg1Tehv708a1m5fDmXPIVXiMiLwGH77xtF5B8qvjJFUV4Wv/2VPdz60R+yc7A83+DwRAyAe67pACxzUWkuDsDGzhDt9X63U9rAVAIRWNde0gi6m4LEs3lgae8D5fLlXKKP/hZ4LTADYIzZC9xZwTUpirIMHB63Hv7f3l8exOeEnG7rawZwtQGAIxNxqnzCho4Q7Q0BphNZjDEMRBL0NNeW9TFwahg1BKuXlMNQLl/OqcyFMWZx3FqhAmtRFGUZyeaLAPz0+EzZuFOzaMuqRnxSrinsH4uyrr2eQHUV7SE/2XyRRDbPyekEaxeZiLqarPwDLXF9ZXEuQmFERF4JGBGpEZEPAYcqvC5FUV4GxaJhLmWZfo5NJcgXiu7cZCxDQ6CahmAN4YaAm7WcWSjwzIkZXrWhHShVO3X6IqxeVLnUqWGkjW+uLM6lSup/wSpp3YOVYfwDNHRUUVY00fQCRQMbO0Icm0qURRhNxjJuM5u2+gAztt/gmRMzZPNF7r7GKh/TbmciD84kmU8t0LdIKDy4tZvRuTS/uKMP5crhXISCGGPeUfGVKIpy3hhjODqZ4JpFyWOztpawtaeJY1MJTs2nXaEwHs24/oC2kJ+ZpLXvj49GCFT7uH1dGwDtISurec/wPMCSZjnrwiH+8i2ai3ClcS7mo5+IyA9E5D1O2QpFUVYG/7FvnNf+7RM8su9U2fis/aC/vscKIXXKWxeLhoGphFu5tK3e7+771LFpbl9X6njmaAZPHrOqEXuroCpXLudS5mIT8MfAdcBuEXlERN5Z8ZUpivKSOOGm//uZobLxxUJhdM4SCiNzKRLZPFu6GwForQ8wm8xRKBoGZ5JsWdXonqMxWMOqpiC7z6ApKFcm5xp99Jwx5nexKprOAl94iUMURbkI7B2ZB6xOaV6OR0rF6xqC1UTsCKMXT1lhqs7Dvy3kJ5HNMzybYqFg3JpGDo5ZqramqqwKqnLlci7Ja40i8m67lebTwDilcteKolxCTkQsYTAVz7LgiTDaPTTPuvZ6muv8tIesfAOAPSPz+Kt8bLJ7HDjVUPeNzgNLtYFru0vCQ6udXh2ci6awF9gGfMQYs8kY89+MMbsquyxFUV6KZDZPPJtnTVsdxpTyDYwxvDA8x02rWwDLb+BEGO0ammNrb5PrN3De/vfYGsdioXDL2lYAV9NQrnzORSisM8Z8AEs4KIpykYnEs3z52SGsQsIlHCHgZCaP287k45EEM8kct661hEJ7KMBMMkuhaNg3FmX76mb3HE4UklPDaHE/5R1rrHMszlFQrlzORSjcrrWPFOXS8V//ZTd/9M0DHLT9AQ5O0tmNvc1AqfeB01v5trVWaGlbyKphNB5Nk8sXWR8u1S9a02Y97PePRelsDBCoLpWxAGgI1vD5X72Ff/o1tRhfLWjtI0VZwRSLxn3IPzUwXTbnCIVt9pv/uN374NmTs3Q2BtwHflsowFwq5zqjvW/9zXV+GoNWutJiJ7PDPdd0aCmLqwitfaQoKxgnighYUu3UEQobOkKEAtWcms9gjOHZEzPctrbNdQyHQ36MKfVQWJyZvKbNqmmkIacKaO0jRVkR7B6e4+9/tLT1+e5hy9a/PlzPyGy6bO7AWIyOhgANgWq6moJMRDOcimaYime5pb/F3a8tZJWreGF4jiqf0N1UXqtoXdgSCq12rSPl6uZchMJ/wap15NQ+2ob2T1aUZcMYw5v/4Wk+9oOjbllrh72jURqD1dyxMczoXMp1NhtjePbkDLetszSC7qYg49G0e7xXG3AijF4Ynqenubas9SbA7z+wme2rm7lvS2clL1O5TDiXjOZpY8w7jDGdxpgOY8w7gf9+EdamKFcFJzyJZz89Ue43OBGxSlL0tdaRzBWYTy0AVkTSZCzrRhKtaqplPJohEreEQrih9NbvFLabSebc3speeppr+cZ7X8Ur1rct63Uplyfn5FM4DW9d1lUoylXMUbvbGVhv816GZlL0t9e79n6nXMWA7Wtwahh1NgWJJLJuBFKZUPCYhTS0VHkpLlQoaGqjoiwTRyetVpd9rbVl5qN0rsB4NMPatnrXDzBhO5edTOZ14VJhO2Ostpk+KfVCAGisLRVDXuxkVpTFnLF0toi0nmkKFQqKct48dWya4dkUv3zb6rLxo5NxVrfW0ddSV9YFzYk8Whuup6XO8gs4jXNORJIEa3x028lnLbbf4OhknNb6AFW+0v+i3vIUqikoL8XZ+insAgynFwC5yixHUa5c3vnZZwG4Y2N72Rv78UiCDeEQTbU1ZYXtnBDSG3qa3RpFc3b10yOTMTZ0hPDZD/9WW2gcnoiftcT1XZvCy3hFypXIGYWCMWbtyzmxiASBJ4CA/T1fN8b8qWf+48CvGWNC9t8B4IvAzViJcr9kjBl8OWtQlJXCfKr0HvXIvnF+8+71ABSKhhPTSe7cFEbEciAbYxAR9o7M01JX4zqH/dU+ZlM5jDEcGIvx4NYu95wt9TUAxDN51pxGG/j8r9wCYmUoK8rZOJfOaxdKFrjXGJMQkRrgKRH5rjHmGRHZAbQs2v89wJwxZoOIvA34K+CXKrg+RbloPH18xt12Mo8Bxuac0hP1JLIFcoUi86kFWur9HDgV5fqeJtf801rnZy5p9UuOphe4blWTe55WT1nr9R31S77/ns0dlbgs5QrkQh3NL4mxcNIxa+yPEZEq4H8Bv7/okDdS6tPwdeA1orV6lSuEJ49FaAhUsy5c72YiQ8lvsD4cosOOGJqKZykWDccjCTZ2lNpsttT7mU0ucGzKila6ttszV1cSCk5EkqJcCJXUFLAFwC5gA/BJY8yzIvLbwLeMMeOLnvk9wAiAMSYvIlGgDZhGUS5zdg7OcevaVnKFIhOxpc7k9eEQhaKVmDYVz1DnryKzUGRjZ+kB31pfw1wqx+B0CoD+tpJG4JTCBtxeCYpyIVRUKBhjCsA2u7fzN0XkTuAXgbsv9Jwi8jDwMMDq1atfYm9FWRlMxjK8cn0byVyBganSe87xSIKWuhpa6v102JFEU7EseVtAeN/6W+r8HDwVY2gmSShQXWYyAnjn7avpbqotMyspyvlyRvORiGwVkWdEZEREPi0iLZ65587nS4wx88BjwD1YWsOAiAwCdSIyYO82BvTZ568GmrArsy4616eNMTuMMTvCYY2kUFYWv/e1vbzrs8+SyObdsWy+QCyTpz0UoKsxyFQ862oFx6eSbinrsMd8NGhHIa1tL2kD4YYAkXiWodkUa9rqlnRC+/M3beV992yo6PUpVz5n8yl8Cvi/ga3AUSxH8Xp77iVDGEQkbGsIiEgtcB+wyxjTZYzpN8b0AyljjPMr/hbwbnv7F4AfmcVdRRRlBROJZ/narlGePDbNf7446Y5P213P2hsCdDYFKRQNM3Z7zOORhCsUQoFq6vxVTMUzDM+mqPOX90XubgqSyOY5MBZzy2IrynJzNvNRgzHme/b2x0RkF/A9EXkXVv7CS9ENfMH2K/iArxpjHjnL/p8FvmRrDrPA287hOxRlxfDYkSl32+tMnrYT0tpDAbeg3WQsS5VPmEnmyvwGHQ0BpuJZsgtFVreWawPdTVZo6nQiy+rWpRFGirIcnNWnICJNxpgogDHmMRF5C/CvwJmynV2MMfuAm15in5BnO4Plb1CUy5K9I/M0BKvJFwyTHmfydMIRCn4303giliGZs0xMGz2O4Y6GIJFYlmh6gdWLtAFvyet+1RSUCnE289FfAdd6B+wH/WuAb1RyUYqykolnFohlFpaM7x+LsrWnie6mIJPxkqZwZNIKIe1oDLo9kSdiGY5NWZFHGz3O5HBjwDUfLe6E1u3pfrZYYCjKcnG2jOZ/drZFJGSPJYwxw8BvXIS1KcqKI5nNc8/Hfkw0nePRD9xFv+0ILhYNh8fjvPuVaygUDRFbUygUDZ9+4gSvXN/GqqYgRQNVPmEqliGWXiAUqC7TADoaAnx7xgo5Xb2ozHWHp/KpNxxVUZaTsyavichvisgwMAQMi8iQiGiDHeWq5Ws7R5hOZFkoGA57Sl5PxbPkCkVWt9XT2VjSFMajaeZTC/zsjasQEap8QjgUYCKa4eik1SvB6zfoaCgJiMXaQE2Vjx998C7+7m3bWKU9k5UKcbYqqX8MvBK42xhzwh5bB/ydiLQaY/78Iq1RUVYM+0aj+Kt95PJFxuZL5SpG56y3+96WWoZnAkzGrH7Jp0s062wMuOaje64pD6v2agOLzUdglcp2ymUrSiU4m6bwLuDNjkAAsLffCjxU6YUpykrkyGSc29a2UuevYmzOU8PIFhB9LbV0NATJLBSJZfIMzlj5Bv3tpQd8Z2OQIxNxphPZJdnHHY0loXC2aqeKUinOJhSMHRG0eDANFCu3JEW5tBSLht/+ygs8cTRSNl4oGgamEmzuaqCnuZax+ZQ7N2T7AVY117oP9kg8w9BMkkC1j06PWajTTmAD2NBZ/tbvmI98ArX+KhTlYnM2oTAmIq9ZPCgi9wLjlVuSolxaDk3E+Pc9p3joc+WJ+6fm02TzRdaHQ/S01HJqvvTO9L0DE2ztaaLOX+1GGE3GsozOpelpqXX7HgB0eRzLizWFjR0hPnT/Jn7yB/dW4tIU5SU5W57CbwH/LiJPYRW1A9gBvAqroqmiXJE8eaxUm2g6kaU9ZL35j8w6UUF1dDQEODxuOZpH51K8OB7jT96wBcAjFDKcimboWeQUduYBVnkEBIDPJ7z/3o3LfEWKcu6cUVMwxhwErsdqlNNvf54ArrfnFOWK5OCpmLvtmIUARmxncl9rHeGGANMJp8S15TfY2mMVovOWwB6fT5eFnAKsarb+vnlNy5L6RYpyqTlb9NEGoNMY87lF468SkQljzPGKr05RLgEjsym33MTYfJqb17TY42l8Ypl/OhqC5IuGuVSOk3b5a8eZXB+oJhSoZnQuRSSRdctTONy2to3PPLSDOze1X9wLU5Rz4Gw+hb8FYqcZj9lzinJFMjqX4rZ1be42gDGGnxyfpr+tnpoqn1vRNJLIcnLaKmUdDpUihzoaA+wbjWIMS8xHVT7hvi2dBKrVkaysPM4mFDqNMfsXD9pj/RVbkaJcJPaMzLNQKA+kS+XyTCdybO5qoKWuxg073TMyzwvD8/zaq63W5a5QiGc5MZ1kbXv9oiQ0SygA9LRooply+XA2odB8ljn9lSuXNU8ei/CmT/6Ev/ru4bLxUVsI9LXWsaq5llN2/sFRu37RnRutZDNHK5iKWZqCt+8BlDuT14W1JIVy+XA2obBTRJbUOBKRX6cUjaQolyWffeokAF9+dhhv245h27Hc11JrO5OtXgjHI0n81T73rd/RFMbm04zNp88oFII15TkKirLSOVtI6u9gtdB8B+UhqX7g5yu8LkVZFgam4mzoKM8FMMaw3zbtpBcKzCZztDlhp94Io1Ap7PREJMHatnq39HV9oJp6fxU7h+YwZqk24EQgtdUHynIUFGWlc7aQ1EljzCuBDwOD9ufDxphXGGMmLs7yFOXC+cHBCX7mr5/g3/eMlY1PxbPMJHO8cr3lTB6PlpLQRmbT1NZYHc/aPWGnpzMRhRsCPHfS6hi7eO4V69tY117PL9zcW4lLU5SKcdYqqWA11zHGfML+/OhiLEpRlgOnE9q/7i4XCgdPWVrCa67tBHD9BgDDsym341k4FCBfNMynFzg1n1niMA43BMgsWI7q/kVC4bpVTfzoQ3fzgfs2Le9FKUqFeUmhoCgrGSt5LHHauedOzgJwaLw8svrgmPX3vZs7AKvhDUAuX+S5kzNs7bWS0By/wcBUgvRCYUkSmjPfHvLTGHzJtuWKclmgQkG5rPnodw7xmv/nxwxOJ8vG84Uiw3ZZiulElly+FHp68FSM/rY61rTW4a/yuTWMnj05QyyT53XXdwGlh/7ekXmAJT0MnOJ1Ws1UuZJQoaCseNK5Ascm40vGjTFuFNHOobmyufFohoWCYfvqZoyBKU97zIPjUa5b1YTPJ7SH/ETsiqWOU3nHGqsFuZN09tygpXEsFgrb+poBuG5V48u9REVZMahQUFY8H/6Pg9z3N08wHk2XjU/aLS8BXhguFwpOHwMnM3nCdiZH0wuMzKbZYj/IW0N+ZpNZ95jmuhqa6ixTUHdTkGqf8Mxxy5m8uHjdm27q4ZH/+mo+dP81y3KdirISUKGgrHietX0D/7prtGzcSSjzCRwYi5bNOeak29Zab/1OhJHjX3De7tvqA8wkrVyEoZkUazwd0qqrfPS21BLP5qn3V7nmJC/X9zTRUu9/eReoKCsIFQrKisYYw6z90N5j2/YdHKHwuq3dHI8ky5LQ9oxEaav3c1OfVcxu0nYmOxVQr1tlOZPb6v3M2AlqQ7NJ1rSW+wccIbEuHNKKpspVgQoFZUVzKpohml4AcB3HDkcn47SH/Ny2tpVENu92MwPYNTTL9jUtNNZWU1tT5WoKB09F6WgIuG/9rfV+ZpM5q+fyXJr+tnKhcG23pVFoFzTlakGFgrKiOWn3Kri+p5Hh2VSZNnB0MsHGjgY22I3sB6as0NRYZoHBmRTb+poREbqbgq5P4cVTMdefANAWCpBeKHB0Mk7RUGY+AnjHbasBuMl2KivKlU7FhIKIBEXkORHZKyIHReTD9vhn7bF9IvJ1EQnZ4wER+T8iMiAiz4pIf6XWplw+nJy2HvR3bQqTWSi6kULGWP2SN3WG6LNNPmN2EpoTqbS5yypv0dUUZDyaJpsvMDCVKIsWarP9AY6j2umJ4NDXWseTv38Pv3u/JqEpVweV1BSywL3GmBuBbcADInI78AFjzI3GmBuAYeD99v7vAeaMMRuAvwH+qoJrUy4TTkwnqa2pYke/5TB2TEgTsQyJbJ4NnQ1lZazB0iCg1P+4qynIZCzLyGyKfNGw0VMLKdxoHes4sxdrCmAJBu19oFwtVEwoGAsn1bTG/hhjTAxALK9dLeDYA94IfMHe/jrwGlHP3lXDQqFIPLOwZHxgKsG6cD399sPaaY85OG39u7atnmBNFQ3Bak++QYw6f5WbZ9DVGGQylnGP7Wst5Rv02YlnTw1MU++vcjUHRblaqahPQUSqRGQPMAU8aox51h7/PDABbAY+Ye/eA4wAGGPyQBRoq+T6lJXDO//xWbb/2aNLxg+Nx7i2u5Ge5lpESprCkJ2HsMZ2DFvtMzMYY3jsSITb1ra61Um7m6zWmS8MzwMlQQDQa9czmk8tsKatXiOMlKueigoFY0zBGLMN6AVuFZHr7fFfBVYBh4BfOp9zisjDIrJTRHZGIpHlXrJyCZiMZXj25CwLBcNMohRBNBXPMJ3IsaW7EX+1j1VNtYw4QmE2RbVP3HpE4YYAU7EsxyNJhmdT3Lelyz1Pl90j+fnBWQLVvrJ8g2BNlVvmerE/QVGuRi5K9JExZh54DHjAM1YAvgK8xR4aA/oARKQaaAJmTnOuTxtjdhhjdoTD4QqvXLkYnIiU6hYdmSiVs3jRzilwwkJXt9Yx5NEUeltqqa6yfsIdDUEiiaxbHG9rT5N7ni674c1zg7P0ttQu0Qac6qd9rSoUFKWS0UdhEWm2t2uB+4AjIrLBHhPg5wCnH+K3gHfb278A/Mh44w+Vy57MQoF0rrBkfNqjHRzyCIVDdi2iLbZQcEJLjTE8d3KOrb3N7r4dtqbgdE5b7XnAd9nahDGw3g5f9fKGG1YB0NusXWYV5Wyd114u3cAXRKQKS/h8Ffg28KSINAIC7AV+097/s8CXRGQAmAXeVsG1KZeAX/n8c+wenufwRx4o60bmOIirfcJhT5nrQ+Mxeppr3VpE4cYAkXiWQ+NxphNZ7tzY7u4bbrDyDV4cj9FUW6pfBJQ5jzd0LBUKv/aqfjZ2hLjVLomhKFczFRMKxph9wE2nmXrVGfbPAL9YqfUol55nTlhhn48fneLezZ3ueCSRpaZKuKW/lcNlmkLMzTUAy0SUKxT5j32nALhjY8l82GGHlu4cmi3TEoAyAXQ6oSAi3LlJTZGKAprRrCwjE9EMf/efx0hm80vmMgsls9GBsfKmN5F4lvZQgC3djRydjFMoGhYKRU5OJ9nY6RUK1oP/67tG2dQZcs1C1py1PTKbXtIaE+Djb7+Jptoatq9ueXkXqShXOJU0HylXGV96ZpBPPnacwxMxPvXOm8vmnFLWUMo1cDgRSRBuCLChI0Q2X+TUfJpsvmgnmpXe7Ds8SWqv39pddg5vRNE1Hu3C4eduXMXP3tCtIaeK8hKopqAsGyftctXfOzjhbjs42kFDsJrh2dLcj49G2D08z7VdjXTab/5T8QwDU5YZaWOnRyg0ljSD7WvK3/g7PEJh82mEAqACQVHOARUKynlhjOEd//gMX1/U2wDg2GSC/rY6jCm1sHT47v5xeppruX9LV5mm8NSxCD6BP/25Le6DfSqWdYvbeaOFejzRQYsL1DXVlhzLW7QTmqJcMCoUlPPi5HSSnwzM8KGv7S0bz+UtH4DjQPY++KOpBZ44FuHBrV30ttQSSWRZKFg9k/eORrmxr5k6fzWdjY6mkOXYVIKe5lrqAyULp7/axz//+m288/bVbiayg4jw5O/fww8+cCfdTRpaqigXivoUlPPiObtwHEC+UHSTx05MJ8gXDTf2NdHdFGTIYyL6wYsTLBQMr79hFQfGohgDs8kc7aEAB8aivHVHHwCtdX6qfcJkLMOxycRpI4VeuaGdV25oXzIOmnymKMuBagrKeeHUD4JSchmUmt5f291oZR57NIVv7x+nt6WWG3ubyiqaDkwlSOUK3NhnZR/7fEJ7KMBENMPxyOmFgqIolUWFgnJeHJqIsS5shXw+NzhbNu6v8rG2vZ7VrXWMzllCIZHN89SxaV6/1Yr8aQ+VhILjd7jBk5nc2RRk59Ac2XyxLPJIUZSLgwoF5ZzJF4ocmYhzzzUd9LXWsmuoJBQOj8fZ0BGipspHd3MtU3HLb3B0Mk6+aNx+CN6w0oOnooQC1az19DDobal1K6F6I48URbk4qFBQlpDNF/jd/7OHJ46WV6Ednk2RzRfZ3NXA2vYQY/MZd+7wRIzN3VYo6KqmIMZY1U+P2hnK19hJaK6mkMhyYjrJunB9Wcaxt6z1hvDpQ0sVRakcKhSUJfz4SIRvvDDGQ597jmy+lIk8Ome1u1zdWkdbvZ+5ZA6wnMaTsSzXdtmF6+zQ0fFohiOTcWprqtxooVp/FQ0BqyHOyenkkuxjbwMcb/0iRVEuDioUrlIGphL0/8G32W33Jvby/YOT7vbIbNrddnog97TU0lJXEgqHJ6zENEdTcHocnJpPc2QizsbOUJk2EG4IMDqXZmx+aUmKTrtcxbZFeQiKolwcVChcpTxiF5X719MkoR08FaXFfksf8pSnGJtLU+UTuhqDtNbXEM/myeYLbuTR5q5SiWuwNIXDE3FXg3Bobwiwa2gWY1giFG5b18rrt3bzibefrpaioiiVRoXCVcpBu4HNrP2275DLFxmYSvDA9VbnskFPaOnIXIquxiDVVT5a7HLU86kFDk/EaA/53XDThmANDYFq9o9GmU3mltQiCjcEmEtZ/ZgXC4WGYA2ffMd2zTlQlEuECoUrmANjUd7490+5Zh8vjsln/1i0bPx4xEpCu31dGw3BaldTmE3mePTFSW7pt2oOtdb53fEjk4klD/7u5iA/OjwFlMxKDuFQqU5R/2kqmiqKculQoXAF89FvH2LvaJSPff9I2XguX2TMdhqPRzPk7ZITUBIWW7ob6WoMMhWzGuD8+OgUqVyB97x6HYCrKcwlcwzPJOlvK3+4dzfVkrbLZW9eZD5yNIpQoJrGoDqTFWUloULhCqVYNK4T+cljEbydTUfnUhQNbF/dTKFomE6UTEiHx+NuElpbyM9MMls2fm23E1pqCYUT00nmUgtLGtusarb8Cp2NAVo9nc+8x3Y2BlAUZWWhQuEKZSqeJZsvsqkzxHQix3i0lFMwZCeH3b6uDYBT0ZJ56ZAdLVRd5aMtFGDGFhjecYDeljp8gpvLsNgHsMouSrepc2muwT2bO3jbLX189t23LNflKoqyTKhQuEJxmto4Tem9voMXbSfzK9dbheUmbIGRyuXZOTjLjXY4aHu9n+mEpSkcm4y7CWgAwZoq+lrrePyIJRQWawpvuqmHh16xhvffs2HJ2joagvzlW25Qf4KirEBUKFyhDNtRQ6+2m9tPxqwH/3Qiy+eeOsm13Y1cZ/cdcLSIHx6y/AY/awuStlCAWCZPZqHAZCyzpFz1+nCInO2PWPyA72ut4yNvvJ7bbG1EUZTLAxUKK4SJaGZJtzLA7VdcLJrTHGX1Ph6ZTS0ZPz6doKZK2NJtPfijdgjoPzx2nJlkjjfc0E1zXQ3BGh/jdnTSjw5P0Vrv57a1Vp2iNtv2f3giTtHgdkZzcGoTtYf8hAJahV1RrgRUKKwQ7vnY49zzscfLHMIHxqJc+yffY+MffZf7/ubHbgaxQ75Q5IG/fYI7/9dj7B8tDy09MBblmq4GgjVV1PmriKYtoXBiOsHa9nred88GRITuplrGYxmKRcMTRyPcubHdzT5usx3EB2zTU/ciofBqu6/B4lwHRVEuX1QorACMMW745vFISVsYmEqQKxR59yvWMDiT4jNPnig7bmg2xeBMCmPgEz86Vna+A2MxtvY0A1arSkcoDM2k3AgigK7GIBPRDBOxDDPJnFvNFEqho47A6WwsFwq32hrFQ6/ofzmXryjKCkKFwgpg2GP+eebEjLs9n7LewH/rNRvpb6tbYl46NllqbPP84KxrYhqPZoimF9xexU21NcynF8gXiozOpVjjySnobg4yPp92HdPeDGNHCOwdnbf2XdTmMlBdxYEPv5Y/ecOWC794RVFWFCoULiIT0YwbzePlyESpg5njEAaIpvOA9VBf1Vy7JDP52KTV3P5tt/Qxl1rgeMT622l67zSpabQ1hfFohoWCob+tFCnU3RRkMp7lhK2hrPHMddjF6Q5PxKn3V7n1kLyEAtVUeYrdKYpyeVMxoSAiQRF5TkT2ishBEfmwPf5lETkiIgdE5HMiUmOPi4h8XEQGRGSfiGyv1NouBbl8kdv/4oc88LdPlPkNAEbs7OJqn5QJjfl0jlCgmuoqH70ttZxaJBR2D8/R31bnVhR1WmA6wmF92BIKzbU1xNILrjbg1RS6mmopFA07B2fxV/nKtAF/tc9NNNvU1YCIPvwV5UqnkppCFrjXGHMjsA14QERuB74MbAa2ArXAr9v7vw7YaH8eBj5VwbVVjD/+t/38+56xJePfPzgBwHQixwt2G0qH0bkU9f4qNnSEiMRLQiGaXqCp1no7X9VUy3QiR8b2PUTTCzw1MM3913W5GcNztrlpYCpBU22N+0B3fApOcTtvSYpVtvP4pydmWN1Wt+St3zEhberQhjeKcjVQMaFgLBL2nzX2xxhjvmPPGeA5oNfe543AF+2pZ4BmEemu1PoqQTpX4H8/M8xvf2XPkrldQ6W+BXuXCIU0vS11hBsCRDwlJ6KpBZptk80qu3GNoy3sH42yUDDctSns7uMVCuvD9e6bfVNtDfOpBYamkwSqfW5LTIAuWyhMxrJlZiV33hYK13arUFCUq4GK+hREpEpE9gBTwKPGmGc9czXAu4Dv2UM9wIjn8FF7bEWxYIeBfm3nyJK5I5Ml34DzRu+wb3SeW/pbaAxWuzZ/B0so1BIOBZg+g6bgvLE7moTzXdd0NRAKVFNTJcwmrQij45EkGzxN7zsaA6QXCuwfi7K6ta6s4Y3XXLSmbWmG8Z+8YQt/97ZtvO3W1We7LYqiXCFUVCgYYwrGmG1Y2sCtInK9Z/ofgCeMMU+ezzlF5GER2SkiOyORyEsfsMwcmYhzeCLO731935K5w+Mxd/uQZ7tQNLw4boWIbugIlQmFQtFwIpKgv72e9oYA04ms63OYT5c0BSc8dCpeKjvRWu+nPRRARGip8zOfyhFNLTCdyLr+BICeZksD2DU0tyQr2es8Pp2m0N9ezxu39RCsqTrHO6QoyuXMRYk+MsbMA48BDwCIyJ8CYeB3PbuNAX2ev3vtscXn+rQxZocxZkc4HK7Yms+E1x8w5YkUAitKx+GUp6n9qfk0mQWrON36cIgTntDSwZkk2XyRa7sbaQ/5yeaLxLNW1NFsMkdTreUX6FgkFA5NxNnUWXrwt9T57d4G1hq8moJTsTRfNEtyDbzO4y2rms71NiiKcoVSyeijsIg029u1wH3AYRH5deC1wNuNMUXPId8CHrKjkG4HosaY8Uqt70I56CksNzJXXl7ixfGYGwY6Nl+ac6KB1oVDdDfXMp3IsmDXDHISwzZ3NdBuN5+ZjmeZTeaYTeZYH7ZMOs11NdRUCZF4lmQ2z4GxKDevaXG/o6Xe8hs8su8U/mofO9aUktB6mkvawWKhAPDXb72Rv/mlG8vOpyjK1UklC9Z0A18QkSos4fNVY8wjIpIHhoCf2m+p3zDGfAT4DvAgMACkgF+t4NoumNG5NKFANYlsntG5NDevscaNMRwej/GGG1cxEc2UaQpODsC6cD3HIwmMsd74uxqDfOwHR1jdWsemzgbXUTydyLkawUa7MqmIEA4FmIpn2DU0R6FouG1tqdhca72fwxNxHjuS5p5rwjR5zELtoaWOZS9v3t67ZExRlKuTigkFY8w+YEn3dWPMab/TjkZ6X6XWcz4UioZ/enqQn7+pZ0mDmFPRNDv6W3j8SITRuVLewHg0QyyT59ruRnYNzpUlmh0Yi9JW76et3u9G80zYHc9G59L8xZu32jkBtqaQyLr5Ct5y1eHGIJF41i2DfdPqZnduTVs93zswQdHgVjl18PkEETBGG9soinJ2NKP5NDxzYoY/e+RF3vOF58vGjTGMz2dYHw7RVu8vEwqOY/nargZWNQfd0FFjDD89McPt69oQEdd8MxnLuBnJTiMaRyhE7Azjen9V2UO8oyFAJJ7l0HiMvtZaGjytLLf2NOEUUu1uLncmA7zF1ga8WoOiKMpitN7xafjJwDQALwzPUywaN4RzPrVAeqHAquZaupqCZSUpHCfzpq4GVjXXssd2SI/OpRmPZrh9nWXjd8w3E9GM24vAcQq31vvxiaUpjM6l6WutK3MEhxsC7B6aY6FQXNL3+HqPk7ineamJ6KM/fz33bu5ga486kxVFOTNXraaQyOb54k8HXYevl2dPzrrb3rITjmawqilIa73f9QGAZSLqa62lMVhDT0stc6kFUrk8B09Zph6nm1lLXQ3+ap+rKXQ0BNxchCqf0FpvaQOjc6kl4aMdDQFmkjmOR5JunwSHvtbSvqtOoykEqqt4cGu3lqpQFOWsXLVC4V2ffZb/8e8Hec4jAMDyJzjmGYBTnt7GTx+3NIhtq5tpqfO7/Q2MMTw/OMstdsRPjyf7+MVTMXxSMhFZJqQAE7EMA5GE26jGYVVzkLH5NCOzKXpbyvMGwp5M5Fs8Ja6d896/pbPs+xVFUc6Xq1IoGGN4YXgegKl4ea7BiUiCVK7AazZbD9gJT1P7x45Msbmrge6mWlrr/W5zmeORBNOJHLfZJiLnTX1sPsOL4zHWh0NlyV9djUHGoxkGJuNsCJcLhd6WWvaPRUnmCvQt6nvsVC0FSzAt5u9/eTuPfuDOMl+DoijK+XBVCoXJWMkkNBUrL2X98Jd2AXD/dZZQcEJL45kFdg7Occ/mDsBKFotl8iwUijxzwtI2brVDRJ0OZY6m4PQ1cOhqsnwOyVyBDZ3lNYV6W+qYt1tnblqkRTgF7qp9ctr2l/5qnxvCqiiKciFclY5mb5mJKU+toWy+wNBMkrs2hXnFujYC1T7GbU3hJwMz5IuGuzdZWdSt9dbb+HxqgWdPztLREHDLRDhv9Ecm4pyKZpbY/7saA+Tyli9jY0f5g99r+rl20XGbuxq5b0snH7x/04VfvKIoylm4KoXCsSkrUigUqC6LIBqeSVE08ObtPXb/4qDrU9g9PIe/ysdNq62s3xZPueo9I3Pc0t/qOnGdPgSPHZkClj7cvVnFGxYJhXXhUlG6xeGjtf4qPvPQjgu/cEVRlJfgqjQfbe1p4r13r2dLdyOP7BsnYdcacmoSOf0GuptqmbCFwt6Rea5d1Yi/2rplrXWWUBiPZhidSy9xGHc0BBmaSSECN/SWh4F6w0LbFiXHvXpDO3/yhi389VtvXK7LVRRFOWeuSqGwo7+V339gs/sg/8LTg0CpHEW/3afY6V+cLxTZPxblRs/DvcNOKnv+5CzGlPc2hlI+wnWrGmmuK3/wb/fUGFocIioivOfVa7X0hKIol4SrUig4fOSN19PTXMtTx6xQ032j8/S11rp5A07/4hdG5knlCmW1hrrsPgROmOq69nJNIZ6xnMV3bVpaybWmyscn3n4TX/7125b/ohRFUV4GV7VQqPIJD27tYtfQHLl8kd3Dc2xfXXqL77b7F3/zhTFE4BXrS0IhFKimIVjNbju0tb+9PHz0wa1W07j3vHrdab/7Z29cxas2tC/zFSmKorw8rkpHs5eNnQ3kCkX2js4zGcuyzc48hpJv4QcHJ+ltqV1SHK+7KUg8kyDcEFiSG/Crr1rLO25b4/ogFEVRLgeu+idWrx0C6tQ78nYsc/IEphPZ02YJOyakxf4EBxUIiqJcblz1Ty2nlMTTAzMArPZkEYc9dYmclpZenBwDbVWpKMqVwlUvFLqagojAc4Oz+AR6PEXoRMTtfHa6yqO/8sp+AO5Q34CiKFcIV71PwV/to7+tnpPTSbqbaqmpKpeTd2wMs3t4fsk4QF9rHXv/x/2Eglf9bVQU5QpBn2bAp965na8+P8ot/Ut7FP+Xu9aTyuV5+22rT3ust+2loijK5Y5YXTAvT3bs2GF27tx5qZehKIpyWSEiu4wxp62Zc9X7FBRFUZQSKhQURVEUFxUKiqIoiosKBUVRFMVFhYKiKIriokJBURRFcVGhoCiKorioUFAURVFcLuvkNRGJAEMX8SvbgemL+H0vha7nzKyktcDKWs9KWgvoes5GpdayxhiztAMYl7lQuNiIyM4zZQFeCnQ9Z2YlrQVW1npW0lpA13M2LsVa1HykKIqiuKhQUBRFUVxUKJwfn77UC1iErufMrKS1wMpaz0paC+h6zsZFX4v6FBRFURQX1RQURVEUFxUKiqIoistVLxRE5HMiMiUiBzxjN4rIT0Vkv4j8h4g0euZusOcO2vNBe/xm++8BEfm4iMglXMvjInJERPbYn45K3xsReYfn+/aISFFEti3XvVnm9bzs+3Oea6kRkS/Y44dE5A89xzxgr2VARP7gQu7LMq9n0B7fIyIX1MHqPNfiF5HP2+N7ReRuzzGX4ndztvUsx++mT0QeE5EX7f9vf9sebxWRR0XkmP1viz0u9rUPiMg+EdnuOde77f2Pici7L+TenBZjzFX9Ae4EtgMHPGPPA3fZ278G/Jm9XQ3sA260/24Dquzt54DbAQG+C7zuEq7lcWDHxbw3i47bChz3/P2y780yr+dl35/z/G/1y8BX7O06YBDoB6qA48A6wA/sBbZcqvXYfw8C7Rfx3rwP+Ly93QHsAnyX6nfzEutZjt9NN7Dd3m4AjgJbgP8J/IE9/gfAX9nbD9rXLva9eNYebwVO2P+22NstL2dtzueq1xSMMU8As4uGNwFP2NuPAm+xt+8H9hlj9trHzhhjCiLSDTQaY54x1n+xLwJvuhRrOd/vXMb1eHk78BWA5bo3y7We5eI812KAehGpBmqBHBADbgUGjDEnjDE5e41vvITrWRbOcy1bgB/Zx00B88COS/i7Oe16LuR7z7CWcWPMbns7DhwCerD+u3/B3u0LlK71jcAXjcUzQLN9b14LPGqMmTXGzNnX8MByrPGqFwpn4CCl/zl/EeiztzcBRkS+LyK7ReT37fEeYNRz/Kg9dinW4vB5W8X9kwtVu89zPV5+CfgXe7uS9+ZC1uNQiftzprV8HUgC48Aw8DFjzCzWfRjxHH+x7s2Z1gOWwPiBiOwSkYcvwlr2Aj8nItUisha42Z67VL+bM63HYdl+NyLSD9wEPAt0GmPG7akJoNPePtNvpGK/HRUKp+fXgPeKyC4sFS9nj1cDrwbeYf/78yLymhW4lncYY7YCd9ifd12E9QAgIrcBKWPMgdMdXAEuZD2Vuj9nWsutQAFYBawFPigi65bpO5d7Pa82xmwHXge8T0TurPBaPof1QNsJ/C3wtL22SnMh61m2342IhIB/BX7HGFOmpdma0SXLFai+VF+8kjHGHMYyzyAim4DX21OjwBPGmGl77jtYtsr/DfR6TtELjF2itfzQGDNmHxsXkX/Gegh8scLrcXgb5W/lY1To3lzgeqjU/TnLWn4Z+J4xZgGYEpGfYJkkRih/C71Y9+ZM6znhuTdTIvJNrHvzxJKTL9NajDF54APOfiLyNJadfY5L8Ls5y3qW7XcjIjVYAuHLxphv2MOTItJtjBm3zUNT9vgYp/+NjAF3Lxp//HzXcjpUUzgNTlSBiPiAPwb+X3vq+8BWEamz7bF3AS/aal9MRG63VcqHgH+/FGux1d52+5ga4A3Asr21n2U9zthb8djvK3lvLmQ9lbw/Z1nLMHCvPVeP5TA8jOXs3Cgia0XEjyXAvrUca7mQ9YhIvYg0eMbvp8L3xv791tvb9wF5Y0xF/5+6kPUs1+/GvpbPAoeMMX/tmfoW4EQQvZvStX4LeEgsbgei9r35PnC/iLSIFal0vz328lkOb/Xl/MF6ixwHFrDevt8D/DbW28FR4C+xM7/t/d+JZY88APxPz/gOe+w48PfeYy7mWoB6rIiJffbc32FHJV2E9dwNPHOa87zse7Nc61mu+3M+awFCwNfs73sR+D3PeR609z8O/NHF+B2faT1YUVB77c/BC13Pea6lHziC5XD9T6ySzpfsd3Om9Szj7+bVWKahfcAe+/MgVvTgD4Fj9ve22vsL8En7HuzHE/2EZQIbsD+/eqG/ncUfLXOhKIqiuKj5SFEURXFRoaAoiqK4qFBQFEVRXFQoKIqiKC4qFBRFURQXFQqKch6ISMEuc3BQrCqaH7Rj3c92TL+I/PLFWqOivBxUKCjK+ZE2xmwzxlwH3IdVDuJPX+KYfqwsYkVZ8WiegqKcByKSMMaEPH+vw8pMbgfWAF/CSnQCeL8x5mkReQa4FjiJVQHz41gJU3cDAeCTxpj/76JdhKKcBRUKinIeLBYK9tg8cA0QB4rGmIyIbAT+xRizQ6xGLR8yxrzB3v9hoMMY8+ciEgB+AvyiMebkRbwURTktWhBPUZaPGuDvxerwVsAqb3467gduEJFfsP9uAjZiaRKKcklRoaAoLwPbfFTAqmr5p8AkcCOWvy5zpsOA/2qMWZ4CZoqyjKijWVEuEBEJY1XX/Htj2WGbgHFjTBGr1n6VvWscq2a/w/eB37SrbSIim5zKnIpyqVFNQVHOj1oR2YNlKspjOZadEsj/APyriDwEfA+ruxlYFTELIrIX+CesCpv9wG67lHKEC2w1qSjLjTqaFUVRFBc1HymKoiguKhQURVEUFxUKiqIoiosKBUVRFMVFhYKiKIriokJBURRFcVGhoCiKorj8/6xlWHsla6IFAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_df\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(train_df['index'], train_df['co2'])\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('CO2 Levels')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run FLAML\n",
    "The AutoML class provides a scikit-learn style estimator (with standard fit and predict functions) for AutoML. In the FLAML automl run configuration, users can specify the task type, time budget, error metric, learner list, whether to subsample, resampling strategy type, and so on. All these arguments have default values which will be used if users do not provide them. For example, the default estimators are `['lgbm', 'rf', 'xgboost', 'extra_tree', 'xgb_limitdepth', 'prophet', 'arima', 'sarimax']`. \n",
    "\n",
    "The documentation of AutoML class can be found here: [Documentation of AutoML](https://microsoft.github.io/FLAML/docs/reference/automl/#automl-objects)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "''' import AutoML class from flaml package '''\n",
    "from flaml import AutoML\n",
    "automl = AutoML()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The AutoML class constructor takes a list of user-specified setting for fitting and prediction. A comprehensive list of setting options available can be found here [List of setting options](https://microsoft.github.io/FLAML/docs/reference/automl/#automl-objects). In particular, users may want to specify a metric for optimization. A list of built-in optimization metrics available (as well as how to customize metrics) can be found at [here](https://microsoft.github.io/FLAML/docs/Use-Cases/Task-Oriented-AutoML/#optimization-metric)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "settings = {\n",
    "    \"time_budget\": 240,  # total running time in seconds\n",
    "    \"metric\": 'mape',  # primary metric for validation: 'mape' is generally used for forecast tasks\n",
    "    \"task\": 'ts_forecast',  # task type\n",
    "    \"log_file_name\": 'CO2_forecast.log',  # flaml log file\n",
    "    \"eval_method\": \"holdout\",  # validation method can be chosen from ['auto', 'holdout', 'cv']\n",
    "    \"seed\": 7654321,  # random seed\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[flaml.automl: 11-07 01:48:11] {2600} INFO - task = ts_forecast\n",
      "[flaml.automl: 11-07 01:48:11] {2602} INFO - Data split method: time\n",
      "[flaml.automl: 11-07 01:48:11] {2605} INFO - Evaluation method: holdout\n",
      "[flaml.automl: 11-07 01:48:11] {2727} INFO - Minimizing error metric: mape\n",
      "[flaml.automl: 11-07 01:48:11] {2869} INFO - List of ML learners in AutoML Run: ['lgbm', 'rf', 'xgboost', 'extra_tree', 'xgb_limitdepth', 'prophet', 'arima', 'sarimax']\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 0, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3297} INFO - Estimated sufficient time budget=146s. Estimated necessary time budget=0s.\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0621,\tbest estimator lgbm's best error=0.0621\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 1, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0621,\tbest estimator lgbm's best error=0.0621\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 2, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0277,\tbest estimator lgbm's best error=0.0277\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 3, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0277,\tbest estimator lgbm's best error=0.0277\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 4, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0175,\tbest estimator lgbm's best error=0.0175\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 5, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0055,\tbest estimator lgbm's best error=0.0055\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 6, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0055,\tbest estimator lgbm's best error=0.0055\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 7, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0055,\tbest estimator lgbm's best error=0.0055\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 8, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:11] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.0031,\tbest estimator lgbm's best error=0.0031\n",
      "[flaml.automl: 11-07 01:48:11] {3164} INFO - iteration 9, current learner rf\n",
      "[flaml.automl: 11-07 01:48:12] {3344} INFO -  at 0.2s,\testimator rf's best error=0.0218,\tbest estimator lgbm's best error=0.0031\n",
      "[flaml.automl: 11-07 01:48:12] {3164} INFO - iteration 10, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:12] {3344} INFO -  at 0.2s,\testimator xgboost's best error=0.6738,\tbest estimator lgbm's best error=0.0031\n",
      "[flaml.automl: 11-07 01:48:12] {3164} INFO - iteration 11, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:12] {3344} INFO -  at 0.2s,\testimator extra_tree's best error=0.0208,\tbest estimator lgbm's best error=0.0031\n",
      "[flaml.automl: 11-07 01:48:12] {3164} INFO - iteration 12, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:12] {3344} INFO -  at 0.3s,\testimator xgb_limitdepth's best error=0.0447,\tbest estimator lgbm's best error=0.0031\n",
      "[flaml.automl: 11-07 01:48:12] {3164} INFO - iteration 13, current learner prophet\n",
      "01:48:12 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:12 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:12] {3344} INFO -  at 0.7s,\testimator prophet's best error=0.0008,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:12] {3164} INFO - iteration 14, current learner arima\n",
      "[flaml.automl: 11-07 01:48:12] {3344} INFO -  at 0.9s,\testimator arima's best error=0.0047,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:12] {3164} INFO - iteration 15, current learner sarimax\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.7s,\testimator sarimax's best error=0.0011,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 16, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.8s,\testimator xgboost's best error=0.6738,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 17, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.8s,\testimator extra_tree's best error=0.0208,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 18, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.8s,\testimator xgboost's best error=0.1709,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 19, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.8s,\testimator xgb_limitdepth's best error=0.0447,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 20, current learner rf\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.8s,\testimator rf's best error=0.0205,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 21, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.8s,\testimator xgb_limitdepth's best error=0.0029,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 22, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator lgbm's best error=0.0031,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 23, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator xgb_limitdepth's best error=0.0029,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 24, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator xgboost's best error=0.0244,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 25, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator xgboost's best error=0.0244,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 26, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator xgboost's best error=0.0244,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 27, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator xgboost's best error=0.0244,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 28, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 1.9s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 29, current learner arima\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.0s,\testimator arima's best error=0.0047,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 30, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.0s,\testimator xgboost's best error=0.0244,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 31, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.0s,\testimator extra_tree's best error=0.0208,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 32, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.0s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 33, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 34, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0027,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 35, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 36, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 37, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 38, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 39, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:13] {3344} INFO -  at 2.1s,\testimator xgboost's best error=0.0030,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:13] {3164} INFO - iteration 40, current learner arima\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.4s,\testimator arima's best error=0.0047,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 41, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.4s,\testimator xgboost's best error=0.0030,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 42, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.4s,\testimator extra_tree's best error=0.0187,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 43, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.4s,\testimator xgboost's best error=0.0026,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 44, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.4s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0008\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 45, current learner prophet\n",
      "01:48:14 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:14 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 46, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:14] {3344} INFO -  at 2.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:14] {3164} INFO - iteration 47, current learner prophet\n",
      "01:48:14 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:14 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 48, current learner sarimax\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.5s,\testimator sarimax's best error=0.0011,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 49, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.5s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 50, current learner rf\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.5s,\testimator rf's best error=0.0205,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 51, current learner prophet\n",
      "01:48:15 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:15 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 52, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 53, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:15] {3344} INFO -  at 3.9s,\testimator extra_tree's best error=0.0097,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:15] {3164} INFO - iteration 54, current learner prophet\n",
      "01:48:15 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:15 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 55, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.4s,\testimator extra_tree's best error=0.0097,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 56, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.4s,\testimator extra_tree's best error=0.0060,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 57, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 58, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 59, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.4s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 60, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.5s,\testimator xgboost's best error=0.0025,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 61, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.5s,\testimator extra_tree's best error=0.0060,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 62, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.5s,\testimator extra_tree's best error=0.0060,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 63, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 64, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 65, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 66, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 67, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 68, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 69, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator extra_tree's best error=0.0045,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 70, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.6s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 71, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.7s,\testimator extra_tree's best error=0.0045,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 72, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.7s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 73, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.7s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 74, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.7s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 75, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:16] {3344} INFO -  at 4.7s,\testimator extra_tree's best error=0.0030,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:16] {3164} INFO - iteration 76, current learner sarimax\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.3s,\testimator sarimax's best error=0.0007,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 77, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.3s,\testimator extra_tree's best error=0.0030,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 78, current learner arima\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.4s,\testimator arima's best error=0.0044,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 79, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 80, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 81, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.5s,\testimator extra_tree's best error=0.0030,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 82, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.5s,\testimator xgboost's best error=0.0025,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 83, current learner lgbm\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 84, current learner prophet\n",
      "01:48:20 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:20 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 85, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.9s,\testimator xgboost's best error=0.0025,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 86, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.9s,\testimator extra_tree's best error=0.0030,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 87, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:20] {3344} INFO -  at 8.9s,\testimator extra_tree's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:20] {3164} INFO - iteration 88, current learner prophet\n",
      "01:48:20 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:20 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 89, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.4s,\testimator extra_tree's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 90, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.4s,\testimator extra_tree's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 91, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.5s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 92, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.5s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 93, current learner prophet\n",
      "01:48:21 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:48:21 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 94, current learner xgboost\n",
      "[flaml.automl: 11-07 01:48:21] {3344} INFO -  at 9.9s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:48:21] {3164} INFO - iteration 95, current learner sarimax\n",
      "[flaml.automl: 11-07 01:50:51] {3344} INFO -  at 159.6s,\testimator sarimax's best error=0.0007,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:51] {3164} INFO - iteration 96, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:51] {3344} INFO -  at 159.7s,\testimator extra_tree's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:51] {3164} INFO - iteration 97, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:51] {3344} INFO -  at 159.7s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:51] {3164} INFO - iteration 98, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:51] {3344} INFO -  at 159.7s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:51] {3164} INFO - iteration 99, current learner prophet\n",
      "01:50:51 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:51 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:51] {3344} INFO -  at 160.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:51] {3164} INFO - iteration 100, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.2s,\testimator rf's best error=0.0173,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 101, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.2s,\testimator rf's best error=0.0097,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 102, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.2s,\testimator rf's best error=0.0097,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 103, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.3s,\testimator rf's best error=0.0044,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 104, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.3s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 105, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.4s,\testimator rf's best error=0.0044,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 106, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 107, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.4s,\testimator rf's best error=0.0044,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 108, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.4s,\testimator rf's best error=0.0041,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 109, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.5s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 110, current learner prophet\n",
      "01:50:52 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:52 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 111, current learner rf\n",
      "[flaml.automl: 11-07 01:50:52] {3344} INFO -  at 160.8s,\testimator rf's best error=0.0041,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:52] {3164} INFO - iteration 112, current learner prophet\n",
      "01:50:52 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:52 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:53] {3344} INFO -  at 161.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:53] {3164} INFO - iteration 113, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:53] {3344} INFO -  at 161.4s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:53] {3164} INFO - iteration 114, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:53] {3344} INFO -  at 161.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:53] {3164} INFO - iteration 115, current learner prophet\n",
      "01:50:53 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:53 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:53] {3344} INFO -  at 161.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:53] {3164} INFO - iteration 116, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:53] {3344} INFO -  at 161.8s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:53] {3164} INFO - iteration 117, current learner sarimax\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.3s,\testimator sarimax's best error=0.0007,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 118, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 119, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 120, current learner prophet\n",
      "01:50:54 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:54 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 121, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.8s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 122, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 123, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.9s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 124, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:54] {3344} INFO -  at 162.9s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:54] {3164} INFO - iteration 125, current learner prophet\n",
      "01:50:54 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:54 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 126, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.3s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 127, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 128, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 129, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 130, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 131, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 132, current learner prophet\n",
      "01:50:55 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:55 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 133, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.8s,\testimator xgboost's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 134, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:55] {3344} INFO -  at 163.8s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:55] {3164} INFO - iteration 135, current learner prophet\n",
      "01:50:55 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:55 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 136, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 137, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 138, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.4s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 139, current learner arima\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.5s,\testimator arima's best error=0.0044,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 140, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.5s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 141, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 142, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.6s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 143, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.6s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 144, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.7s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 145, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.7s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 146, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.7s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 147, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 148, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 149, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 150, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 151, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 152, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 153, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 154, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.9s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 155, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.9s,\testimator rf's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 156, current learner rf\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 164.9s,\testimator rf's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 157, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 165.0s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 158, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 165.0s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 159, current learner arima\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 165.1s,\testimator arima's best error=0.0043,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 160, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 165.1s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 161, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 165.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 162, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:56] {3344} INFO -  at 165.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:56] {3164} INFO - iteration 163, current learner prophet\n",
      "01:50:56 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:57 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:57] {3344} INFO -  at 165.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:57] {3164} INFO - iteration 164, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:57] {3344} INFO -  at 165.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:57] {3164} INFO - iteration 165, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:57] {3344} INFO -  at 165.6s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:57] {3164} INFO - iteration 166, current learner arima\n",
      "[flaml.automl: 11-07 01:50:57] {3344} INFO -  at 166.1s,\testimator arima's best error=0.0033,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:57] {3164} INFO - iteration 167, current learner arima\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 166.6s,\testimator arima's best error=0.0033,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 168, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 166.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 169, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 166.7s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 170, current learner prophet\n",
      "01:50:58 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:58 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 167.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 171, current learner arima\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 167.1s,\testimator arima's best error=0.0033,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 172, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 167.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 173, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 167.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 174, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 167.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 175, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:58] {3344} INFO -  at 167.1s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:58] {3164} INFO - iteration 176, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.2s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 177, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.2s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 178, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.2s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 179, current learner arima\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.6s,\testimator arima's best error=0.0033,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 180, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.7s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 181, current learner lgbm\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 182, current learner xgboost\n",
      "[flaml.automl: 11-07 01:50:59] {3344} INFO -  at 167.8s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:50:59] {3164} INFO - iteration 183, current learner prophet\n",
      "01:50:59 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:50:59 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 184, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.2s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 185, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.2s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 186, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.2s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 187, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.2s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 188, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.3s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 189, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 190, current learner prophet\n",
      "01:51:00 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:00 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 191, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 192, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 168.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 193, current learner prophet\n",
      "01:51:00 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:00 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 169.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 194, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 169.0s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 195, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 169.1s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 196, current learner rf\n",
      "[flaml.automl: 11-07 01:51:00] {3344} INFO -  at 169.1s,\testimator rf's best error=0.0021,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:00] {3164} INFO - iteration 197, current learner prophet\n",
      "01:51:01 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:01 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:01] {3344} INFO -  at 169.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:01] {3164} INFO - iteration 198, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:01] {3344} INFO -  at 169.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:01] {3164} INFO - iteration 199, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:01] {3344} INFO -  at 169.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:01] {3164} INFO - iteration 200, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:01] {3344} INFO -  at 169.5s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:01] {3164} INFO - iteration 201, current learner prophet\n",
      "01:51:01 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:01 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:01] {3344} INFO -  at 169.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:01] {3164} INFO - iteration 202, current learner prophet\n",
      "01:51:01 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:01 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:02] {3344} INFO -  at 170.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:02] {3164} INFO - iteration 203, current learner prophet\n",
      "01:51:02 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:02 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:02] {3344} INFO -  at 170.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:02] {3164} INFO - iteration 204, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:02] {3344} INFO -  at 170.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:02] {3164} INFO - iteration 205, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:02] {3344} INFO -  at 170.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:02] {3164} INFO - iteration 206, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:02] {3344} INFO -  at 170.6s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:02] {3164} INFO - iteration 207, current learner prophet\n",
      "01:51:02 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:02 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:02] {3344} INFO -  at 171.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:02] {3164} INFO - iteration 208, current learner prophet\n",
      "01:51:02 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:03 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 171.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 209, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 171.4s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 210, current learner prophet\n",
      "01:51:03 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:03 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 171.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 211, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 171.9s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 212, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 172.0s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 213, current learner rf\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 172.0s,\testimator rf's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 214, current learner rf\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 172.0s,\testimator rf's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 215, current learner rf\n",
      "[flaml.automl: 11-07 01:51:03] {3344} INFO -  at 172.1s,\testimator rf's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:03] {3164} INFO - iteration 216, current learner prophet\n",
      "01:51:03 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:04 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:04] {3344} INFO -  at 172.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:04] {3164} INFO - iteration 217, current learner prophet\n",
      "01:51:04 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:04 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:04] {3344} INFO -  at 172.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:04] {3164} INFO - iteration 218, current learner rf\n",
      "[flaml.automl: 11-07 01:51:04] {3344} INFO -  at 172.9s,\testimator rf's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:04] {3164} INFO - iteration 219, current learner prophet\n",
      "01:51:04 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:04 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 220, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 221, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 222, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 223, current learner rf\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.3s,\testimator rf's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 224, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 225, current learner prophet\n",
      "01:51:05 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:05 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 226, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 227, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 228, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 229, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 230, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 231, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.8s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 232, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:05] {3344} INFO -  at 173.9s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:05] {3164} INFO - iteration 233, current learner prophet\n",
      "01:51:05 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:05 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 234, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.3s,\testimator xgb_limitdepth's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 235, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.3s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 236, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.3s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 237, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.3s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 238, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.3s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 239, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.4s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 240, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.4s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 241, current learner rf\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.4s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 242, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.4s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 243, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.5s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 244, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 245, current learner rf\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.5s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 246, current learner rf\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.5s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 247, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.6s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 248, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 174.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 249, current learner prophet\n",
      "01:51:06 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:06 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 250, current learner rf\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.0s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 251, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.0s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 252, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.1s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 253, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.1s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 254, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 255, current learner rf\n",
      "[flaml.automl: 11-07 01:51:06] {3344} INFO -  at 175.1s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:06] {3164} INFO - iteration 256, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:07] {3344} INFO -  at 175.2s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:07] {3164} INFO - iteration 257, current learner prophet\n",
      "01:51:07 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:07 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:07] {3344} INFO -  at 175.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:07] {3164} INFO - iteration 258, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:07] {3344} INFO -  at 175.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:07] {3164} INFO - iteration 259, current learner prophet\n",
      "01:51:07 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:07 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:07] {3344} INFO -  at 175.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:07] {3164} INFO - iteration 260, current learner rf\n",
      "[flaml.automl: 11-07 01:51:07] {3344} INFO -  at 176.0s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:07] {3164} INFO - iteration 261, current learner prophet\n",
      "01:51:07 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:08 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 262, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.5s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 263, current learner prophet\n",
      "01:51:08 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:08 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 264, current learner rf\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.8s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 265, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.9s,\testimator xgboost's best error=0.0020,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 266, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 267, current learner rf\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 176.9s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 268, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:08] {3344} INFO -  at 177.0s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:08] {3164} INFO - iteration 269, current learner prophet\n",
      "01:51:08 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:09 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 270, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.5s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 271, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.5s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 272, current learner prophet\n",
      "01:51:09 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:09 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 273, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 274, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.9s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 275, current learner rf\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.9s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 276, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 177.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 277, current learner rf\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 178.0s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 278, current learner rf\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 178.0s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 279, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 178.0s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 280, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 178.1s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 281, current learner rf\n",
      "[flaml.automl: 11-07 01:51:09] {3344} INFO -  at 178.1s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:09] {3164} INFO - iteration 282, current learner prophet\n",
      "01:51:09 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:10 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:10] {3344} INFO -  at 178.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:10] {3164} INFO - iteration 283, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:10] {3344} INFO -  at 178.5s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:10] {3164} INFO - iteration 284, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:10] {3344} INFO -  at 178.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:10] {3164} INFO - iteration 285, current learner rf\n",
      "[flaml.automl: 11-07 01:51:10] {3344} INFO -  at 178.6s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:10] {3164} INFO - iteration 286, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:10] {3344} INFO -  at 178.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:10] {3164} INFO - iteration 287, current learner prophet\n",
      "01:51:10 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:10 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:10] {3344} INFO -  at 178.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:10] {3164} INFO - iteration 288, current learner prophet\n",
      "01:51:10 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:10 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 179.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 289, current learner prophet\n",
      "01:51:11 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:11 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 179.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 290, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 179.7s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 291, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 179.7s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 292, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 179.7s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 293, current learner prophet\n",
      "01:51:11 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:11 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 180.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 294, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:11] {3344} INFO -  at 180.1s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:11] {3164} INFO - iteration 295, current learner prophet\n",
      "01:51:12 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:12 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 180.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 296, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 180.5s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 297, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 180.5s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 298, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 180.6s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 299, current learner prophet\n",
      "01:51:12 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:12 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 300, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.0s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 301, current learner rf\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.1s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 302, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 303, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 304, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 305, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:12] {3344} INFO -  at 181.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:12] {3164} INFO - iteration 306, current learner arima\n",
      "[flaml.automl: 11-07 01:51:13] {3344} INFO -  at 181.6s,\testimator arima's best error=0.0033,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:13] {3164} INFO - iteration 307, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:13] {3344} INFO -  at 181.6s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:13] {3164} INFO - iteration 308, current learner prophet\n",
      "01:51:13 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:13 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:13] {3344} INFO -  at 181.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:13] {3164} INFO - iteration 309, current learner prophet\n",
      "01:51:13 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:13 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 310, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.4s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 311, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.4s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 312, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 313, current learner rf\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.4s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 314, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 315, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 316, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.5s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 317, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 318, current learner prophet\n",
      "01:51:14 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:14 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:14] {3344} INFO -  at 182.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:14] {3164} INFO - iteration 319, current learner prophet\n",
      "01:51:14 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:14 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 320, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.2s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 321, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.2s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 322, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 323, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.3s,\testimator xgboost's best error=0.0019,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 324, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.3s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 325, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 326, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 327, current learner prophet\n",
      "01:51:15 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:15 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 328, current learner rf\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.8s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 329, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.8s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 330, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 331, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.8s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 332, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.9s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 333, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.9s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 334, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 183.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 335, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 184.0s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 336, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 184.0s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 337, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 184.0s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 338, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:15] {3344} INFO -  at 184.1s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:15] {3164} INFO - iteration 339, current learner prophet\n",
      "01:51:15 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:16 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:16] {3344} INFO -  at 184.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:16] {3164} INFO - iteration 340, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:16] {3344} INFO -  at 184.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:16] {3164} INFO - iteration 341, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:16] {3344} INFO -  at 184.5s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:16] {3164} INFO - iteration 342, current learner prophet\n",
      "01:51:16 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:16 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:16] {3344} INFO -  at 184.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:16] {3164} INFO - iteration 343, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:16] {3344} INFO -  at 184.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:16] {3164} INFO - iteration 344, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:16] {3344} INFO -  at 184.9s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:16] {3164} INFO - iteration 345, current learner prophet\n",
      "01:51:16 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:16 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:17] {3344} INFO -  at 185.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:17] {3164} INFO - iteration 346, current learner prophet\n",
      "01:51:17 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:17 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:17] {3344} INFO -  at 185.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:17] {3164} INFO - iteration 347, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:17] {3344} INFO -  at 185.7s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:17] {3164} INFO - iteration 348, current learner prophet\n",
      "01:51:17 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:17 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:17] {3344} INFO -  at 186.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:17] {3164} INFO - iteration 349, current learner prophet\n",
      "01:51:17 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:18 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:18] {3344} INFO -  at 186.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:18] {3164} INFO - iteration 350, current learner prophet\n",
      "01:51:18 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:18 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:18] {3344} INFO -  at 186.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:18] {3164} INFO - iteration 351, current learner prophet\n",
      "01:51:18 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:18 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 352, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.3s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 353, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.3s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 354, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.3s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 355, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.3s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 356, current learner prophet\n",
      "01:51:19 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:19 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 357, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 187.7s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 358, current learner prophet\n",
      "01:51:19 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:19 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:19] {3344} INFO -  at 188.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:19] {3164} INFO - iteration 359, current learner prophet\n",
      "01:51:19 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:20 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 360, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 361, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.4s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 362, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.5s,\testimator extra_tree's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 363, current learner lgbm\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.5s,\testimator lgbm's best error=0.0022,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 364, current learner prophet\n",
      "01:51:20 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:20 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 365, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:20] {3344} INFO -  at 188.9s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:20] {3164} INFO - iteration 366, current learner prophet\n",
      "01:51:20 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:20 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:21] {3344} INFO -  at 189.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:21] {3164} INFO - iteration 367, current learner sarimax\n",
      "[flaml.automl: 11-07 01:51:44] {3344} INFO -  at 212.2s,\testimator sarimax's best error=0.0007,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:44] {3164} INFO - iteration 368, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:44] {3344} INFO -  at 212.2s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:44] {3164} INFO - iteration 369, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:44] {3344} INFO -  at 212.2s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:44] {3164} INFO - iteration 370, current learner prophet\n",
      "01:51:44 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:44 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:44] {3344} INFO -  at 212.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:44] {3164} INFO - iteration 371, current learner prophet\n",
      "01:51:44 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:44 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:44] {3344} INFO -  at 213.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:44] {3164} INFO - iteration 372, current learner arima\n",
      "[flaml.automl: 11-07 01:51:44] {3344} INFO -  at 213.1s,\testimator arima's best error=0.0033,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:44] {3164} INFO - iteration 373, current learner prophet\n",
      "01:51:45 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:45 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:45] {3344} INFO -  at 213.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:45] {3164} INFO - iteration 374, current learner sarimax\n",
      "[flaml.automl: 11-07 01:51:46] {3344} INFO -  at 214.9s,\testimator sarimax's best error=0.0007,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:46] {3164} INFO - iteration 375, current learner prophet\n",
      "01:51:46 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:46 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:47] {3344} INFO -  at 215.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:47] {3164} INFO - iteration 376, current learner prophet\n",
      "01:51:47 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:47 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:47] {3344} INFO -  at 215.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:47] {3164} INFO - iteration 377, current learner xgboost\n",
      "[flaml.automl: 11-07 01:51:47] {3344} INFO -  at 215.7s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:47] {3164} INFO - iteration 378, current learner prophet\n",
      "01:51:47 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:47 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:47] {3344} INFO -  at 216.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:47] {3164} INFO - iteration 379, current learner rf\n",
      "[flaml.automl: 11-07 01:51:47] {3344} INFO -  at 216.1s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:47] {3164} INFO - iteration 380, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:48] {3344} INFO -  at 216.2s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:48] {3164} INFO - iteration 381, current learner prophet\n",
      "01:51:48 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:48 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:48] {3344} INFO -  at 216.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:48] {3164} INFO - iteration 382, current learner rf\n",
      "[flaml.automl: 11-07 01:51:48] {3344} INFO -  at 216.5s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:48] {3164} INFO - iteration 383, current learner prophet\n",
      "01:51:48 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:48 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:48] {3344} INFO -  at 217.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:48] {3164} INFO - iteration 384, current learner prophet\n",
      "01:51:48 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:48 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:49] {3344} INFO -  at 217.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:49] {3164} INFO - iteration 385, current learner prophet\n",
      "01:51:49 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:49 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:49] {3344} INFO -  at 217.7s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:49] {3164} INFO - iteration 386, current learner prophet\n",
      "01:51:49 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:49 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:49] {3344} INFO -  at 218.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:49] {3164} INFO - iteration 387, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:51:49] {3344} INFO -  at 218.1s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:49] {3164} INFO - iteration 388, current learner prophet\n",
      "01:51:49 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:50 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:50] {3344} INFO -  at 218.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:50] {3164} INFO - iteration 389, current learner prophet\n",
      "01:51:50 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:51:50 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:51:50] {3344} INFO -  at 219.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:51:50] {3164} INFO - iteration 390, current learner sarimax\n",
      "[flaml.automl: 11-07 01:52:03] {3344} INFO -  at 231.8s,\testimator sarimax's best error=0.0007,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:03] {3164} INFO - iteration 391, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:52:03] {3344} INFO -  at 231.9s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:03] {3164} INFO - iteration 392, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:52:03] {3344} INFO -  at 231.9s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:03] {3164} INFO - iteration 393, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:52:03] {3344} INFO -  at 231.9s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:03] {3164} INFO - iteration 394, current learner prophet\n",
      "01:52:03 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:03 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:04] {3344} INFO -  at 232.2s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:04] {3164} INFO - iteration 395, current learner prophet\n",
      "01:52:04 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:04 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:04] {3344} INFO -  at 232.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:04] {3164} INFO - iteration 396, current learner xgboost\n",
      "[flaml.automl: 11-07 01:52:04] {3344} INFO -  at 232.6s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:04] {3164} INFO - iteration 397, current learner prophet\n",
      "01:52:04 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:04 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:04] {3344} INFO -  at 233.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:04] {3164} INFO - iteration 398, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:52:04] {3344} INFO -  at 233.1s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:04] {3164} INFO - iteration 399, current learner prophet\n",
      "01:52:04 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:05 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:05] {3344} INFO -  at 233.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:05] {3164} INFO - iteration 400, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:52:05] {3344} INFO -  at 233.5s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:05] {3164} INFO - iteration 401, current learner rf\n",
      "[flaml.automl: 11-07 01:52:05] {3344} INFO -  at 233.6s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:05] {3164} INFO - iteration 402, current learner prophet\n",
      "01:52:05 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:05 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:05] {3344} INFO -  at 234.0s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:05] {3164} INFO - iteration 403, current learner prophet\n",
      "01:52:05 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:05 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:06] {3344} INFO -  at 234.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:06] {3164} INFO - iteration 404, current learner prophet\n",
      "01:52:06 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:06 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:06] {3344} INFO -  at 234.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:06] {3164} INFO - iteration 405, current learner prophet\n",
      "01:52:06 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:06 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:06] {3344} INFO -  at 235.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:06] {3164} INFO - iteration 406, current learner prophet\n",
      "01:52:06 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:07 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:07] {3344} INFO -  at 235.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:07] {3164} INFO - iteration 407, current learner prophet\n",
      "01:52:07 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:07 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:07] {3344} INFO -  at 235.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:07] {3164} INFO - iteration 408, current learner prophet\n",
      "01:52:07 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:07 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:08] {3344} INFO -  at 236.3s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:08] {3164} INFO - iteration 409, current learner prophet\n",
      "01:52:08 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:08 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:08] {3344} INFO -  at 236.6s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:08] {3164} INFO - iteration 410, current learner rf\n",
      "[flaml.automl: 11-07 01:52:08] {3344} INFO -  at 236.7s,\testimator rf's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:08] {3164} INFO - iteration 411, current learner prophet\n",
      "01:52:08 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:08 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:08] {3344} INFO -  at 237.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:08] {3164} INFO - iteration 412, current learner prophet\n",
      "01:52:09 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:09 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 237.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 413, current learner prophet\n",
      "01:52:09 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:09 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 237.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 414, current learner xgboost\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 237.9s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 415, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 237.9s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 416, current learner xgboost\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 238.0s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 417, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 238.0s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 418, current learner xgboost\n",
      "[flaml.automl: 11-07 01:52:09] {3344} INFO -  at 238.0s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:09] {3164} INFO - iteration 419, current learner prophet\n",
      "01:52:09 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:10 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:10] {3344} INFO -  at 238.4s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:10] {3164} INFO - iteration 420, current learner prophet\n",
      "01:52:10 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:10 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:10] {3344} INFO -  at 238.8s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:10] {3164} INFO - iteration 421, current learner prophet\n",
      "01:52:10 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:10 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:10] {3344} INFO -  at 239.1s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:10] {3164} INFO - iteration 422, current learner prophet\n",
      "01:52:10 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:11 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:11] {3344} INFO -  at 239.5s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:11] {3164} INFO - iteration 423, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:52:11] {3344} INFO -  at 239.6s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:11] {3164} INFO - iteration 424, current learner prophet\n",
      "01:52:11 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:11 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:11] {3344} INFO -  at 239.9s,\testimator prophet's best error=0.0005,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:11] {3164} INFO - iteration 425, current learner xgboost\n",
      "[flaml.automl: 11-07 01:52:11] {3344} INFO -  at 239.9s,\testimator xgboost's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:11] {3164} INFO - iteration 426, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:52:11] {3344} INFO -  at 240.0s,\testimator extra_tree's best error=0.0017,\tbest estimator prophet's best error=0.0005\n",
      "[flaml.automl: 11-07 01:52:11] {3164} INFO - iteration 427, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:52:11] {3344} INFO -  at 240.0s,\testimator xgb_limitdepth's best error=0.0018,\tbest estimator prophet's best error=0.0005\n",
      "01:52:11 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:52:12 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:52:12] {3608} INFO - retrain prophet for 0.3s\n",
      "[flaml.automl: 11-07 01:52:12] {3615} INFO - retrained model: <prophet.forecaster.Prophet object at 0x7f73637cbbe0>\n",
      "[flaml.automl: 11-07 01:52:12] {2900} INFO - fit succeeded\n",
      "[flaml.automl: 11-07 01:52:12] {2901} INFO - Time taken to find the best model: 215.2748110294342\n",
      "[flaml.automl: 11-07 01:52:12] {2912} WARNING - Time taken to find the best model is 90% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.\n"
     ]
    }
   ],
   "source": [
    "'''The main flaml automl API'''\n",
    "automl.fit(dataframe=train_df,  # training data\n",
    "           label='co2',  # label column\n",
    "           period=time_horizon,  # key word argument 'period' must be included for forecast task)\n",
    "           **settings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best model and metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best ML leaner: prophet\n",
      "Best hyperparmeter config: {'changepoint_prior_scale': 0.03231895576237737, 'seasonality_prior_scale': 8.339815860996497, 'holidays_prior_scale': 10.0, 'seasonality_mode': 'additive'}\n",
      "Best mape on validation data: 0.00047591896091656326\n",
      "Training duration of best run: 0.269672155380249s\n"
     ]
    }
   ],
   "source": [
    "''' retrieve best config and best learner'''\n",
    "print('Best ML leaner:', automl.best_estimator)\n",
    "print('Best hyperparmeter config:', automl.best_config)\n",
    "print(f'Best mape on validation data: {automl.best_loss}')\n",
    "print(f'Training duration of best run: {automl.best_config_train_time}s')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<prophet.forecaster.Prophet at 0x7f73637cbbe0>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "automl.model.estimator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "''' pickle and save the automl object '''\n",
    "import pickle\n",
    "with open('automl.pkl', 'wb') as f:\n",
    "    pickle.dump(automl, f, pickle.HIGHEST_PROTOCOL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted labels\n",
      "0     370.443824\n",
      "1     371.170715\n",
      "2     372.223428\n",
      "3     373.414165\n",
      "4     373.908790\n",
      "5     373.399986\n",
      "6     372.046985\n",
      "7     370.141438\n",
      "8     368.558874\n",
      "9     368.637837\n",
      "10    369.854784\n",
      "11    371.127363\n",
      "Name: yhat, dtype: float64\n",
      "True labels\n",
      "514    370.175\n",
      "515    371.325\n",
      "516    372.060\n",
      "517    372.775\n",
      "518    373.800\n",
      "519    373.060\n",
      "520    371.300\n",
      "521    369.425\n",
      "522    367.880\n",
      "523    368.050\n",
      "524    369.375\n",
      "525    371.020\n",
      "Name: co2, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "''' compute predictions of testing dataset '''\n",
    "flaml_y_pred = automl.predict(X_test)\n",
    "print(f\"Predicted labels\\n{flaml_y_pred}\")\n",
    "print(f\"True labels\\n{y_test}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mape = 0.001123325711020356\n"
     ]
    }
   ],
   "source": [
    "''' compute different metric values on testing dataset'''\n",
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print('mape', '=', sklearn_metric_loss_score('mape', y_true=y_test, y_predict=flaml_y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Log history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Current Learner': 'lgbm', 'Current Sample': 502, 'Current Hyper-parameters': {'n_estimators': 4, 'num_leaves': 4, 'min_child_samples': 20, 'learning_rate': 0.09999999999999995, 'log_max_bin': 8, 'colsample_bytree': 1.0, 'reg_alpha': 0.0009765625, 'reg_lambda': 1.0, 'optimize_for_horizon': False, 'lags': 3}, 'Best Learner': 'lgbm', 'Best Hyper-parameters': {'n_estimators': 4, 'num_leaves': 4, 'min_child_samples': 20, 'learning_rate': 0.09999999999999995, 'log_max_bin': 8, 'colsample_bytree': 1.0, 'reg_alpha': 0.0009765625, 'reg_lambda': 1.0, 'optimize_for_horizon': False, 'lags': 3}}\n",
      "{'Current Learner': 'lgbm', 'Current Sample': 502, 'Current Hyper-parameters': {'n_estimators': 8, 'num_leaves': 4, 'min_child_samples': 19, 'learning_rate': 0.18686130359903158, 'log_max_bin': 9, 'colsample_bytree': 0.9311834484407709, 'reg_alpha': 0.0013872402855481538, 'reg_lambda': 0.43503398494225104, 'optimize_for_horizon': False, 'lags': 1}, 'Best Learner': 'lgbm', 'Best Hyper-parameters': {'n_estimators': 8, 'num_leaves': 4, 'min_child_samples': 19, 'learning_rate': 0.18686130359903158, 'log_max_bin': 9, 'colsample_bytree': 0.9311834484407709, 'reg_alpha': 0.0013872402855481538, 'reg_lambda': 0.43503398494225104, 'optimize_for_horizon': False, 'lags': 1}}\n",
      "{'Current Learner': 'lgbm', 'Current Sample': 502, 'Current Hyper-parameters': {'n_estimators': 9, 'num_leaves': 4, 'min_child_samples': 14, 'learning_rate': 0.23100120527451992, 'log_max_bin': 8, 'colsample_bytree': 1.0, 'reg_alpha': 0.0009765625, 'reg_lambda': 0.028424597762235913, 'optimize_for_horizon': False, 'lags': 1}, 'Best Learner': 'lgbm', 'Best Hyper-parameters': {'n_estimators': 9, 'num_leaves': 4, 'min_child_samples': 14, 'learning_rate': 0.23100120527451992, 'log_max_bin': 8, 'colsample_bytree': 1.0, 'reg_alpha': 0.0009765625, 'reg_lambda': 0.028424597762235913, 'optimize_for_horizon': False, 'lags': 1}}\n",
      "{'Current Learner': 'lgbm', 'Current Sample': 502, 'Current Hyper-parameters': {'n_estimators': 9, 'num_leaves': 9, 'min_child_samples': 9, 'learning_rate': 0.2917244979615619, 'log_max_bin': 7, 'colsample_bytree': 1.0, 'reg_alpha': 0.0009765625, 'reg_lambda': 0.006048554644106909, 'optimize_for_horizon': False, 'lags': 4}, 'Best Learner': 'lgbm', 'Best Hyper-parameters': {'n_estimators': 9, 'num_leaves': 9, 'min_child_samples': 9, 'learning_rate': 0.2917244979615619, 'log_max_bin': 7, 'colsample_bytree': 1.0, 'reg_alpha': 0.0009765625, 'reg_lambda': 0.006048554644106909, 'optimize_for_horizon': False, 'lags': 4}}\n",
      "{'Current Learner': 'lgbm', 'Current Sample': 502, 'Current Hyper-parameters': {'n_estimators': 4, 'num_leaves': 8, 'min_child_samples': 11, 'learning_rate': 0.8116893577982964, 'log_max_bin': 8, 'colsample_bytree': 0.97502360023323, 'reg_alpha': 0.0012398377555843262, 'reg_lambda': 0.02776044509327881, 'optimize_for_horizon': False, 'lags': 4}, 'Best Learner': 'lgbm', 'Best Hyper-parameters': {'n_estimators': 4, 'num_leaves': 8, 'min_child_samples': 11, 'learning_rate': 0.8116893577982964, 'log_max_bin': 8, 'colsample_bytree': 0.97502360023323, 'reg_alpha': 0.0012398377555843262, 'reg_lambda': 0.02776044509327881, 'optimize_for_horizon': False, 'lags': 4}}\n",
      "{'Current Learner': 'prophet', 'Current Sample': 502, 'Current Hyper-parameters': {'changepoint_prior_scale': 0.05, 'seasonality_prior_scale': 10.0, 'holidays_prior_scale': 10.0, 'seasonality_mode': 'multiplicative'}, 'Best Learner': 'prophet', 'Best Hyper-parameters': {'changepoint_prior_scale': 0.05, 'seasonality_prior_scale': 10.0, 'holidays_prior_scale': 10.0, 'seasonality_mode': 'multiplicative'}}\n",
      "{'Current Learner': 'prophet', 'Current Sample': 502, 'Current Hyper-parameters': {'changepoint_prior_scale': 0.02574943279263944, 'seasonality_prior_scale': 10.0, 'holidays_prior_scale': 10.0, 'seasonality_mode': 'additive'}, 'Best Learner': 'prophet', 'Best Hyper-parameters': {'changepoint_prior_scale': 0.02574943279263944, 'seasonality_prior_scale': 10.0, 'holidays_prior_scale': 10.0, 'seasonality_mode': 'additive'}}\n",
      "{'Current Learner': 'prophet', 'Current Sample': 502, 'Current Hyper-parameters': {'changepoint_prior_scale': 0.029044518309983725, 'seasonality_prior_scale': 10.0, 'holidays_prior_scale': 8.831739687246309, 'seasonality_mode': 'additive'}, 'Best Learner': 'prophet', 'Best Hyper-parameters': {'changepoint_prior_scale': 0.029044518309983725, 'seasonality_prior_scale': 10.0, 'holidays_prior_scale': 8.831739687246309, 'seasonality_mode': 'additive'}}\n",
      "{'Current Learner': 'prophet', 'Current Sample': 502, 'Current Hyper-parameters': {'changepoint_prior_scale': 0.024675775800707445, 'seasonality_prior_scale': 7.131966947593234, 'holidays_prior_scale': 9.840267828793548, 'seasonality_mode': 'additive'}, 'Best Learner': 'prophet', 'Best Hyper-parameters': {'changepoint_prior_scale': 0.024675775800707445, 'seasonality_prior_scale': 7.131966947593234, 'holidays_prior_scale': 9.840267828793548, 'seasonality_mode': 'additive'}}\n"
     ]
    }
   ],
   "source": [
    "from flaml.automl.data import get_output_from_log\n",
    "time_history, best_valid_loss_history, valid_loss_history, config_history, train_loss_history = \\\n",
    "    get_output_from_log(filename=settings['log_file_name'], time_budget=180)\n",
    "\n",
    "for config in config_history:\n",
    "    print(config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAiyklEQVR4nO3dfZxdVX3v8c/XIUCUhwAZKSSQxALRWCjBEUTEANUGqEIIqEBrFVvRKt5aSiy5VKRwKdigvfqSykXLhXjlMYUYbTRQHqs8ZWBIQsDBEBAyQQiGIA+RkOR3/9jrhJ2TfSY7ZPacMzPf9+t1XrP32muf/ZudyfmdtdfeaykiMDMzq/eWZgdgZmatyQnCzMwKOUGYmVkhJwgzMyvkBGFmZoWcIMzMrJAThNmbIOlwSd3NjsOsSk4QNuBIelLSh5oZQ0T8d0SMr+r9JU2WdJeklyStkHSnpOOqOp5ZEScIswKS2pp47JOAG4CZwGhgd+Bc4KNv4r0kyf/P7U3xH44NGpLeIulsSY9L+q2k6yXtmtt+g6TfSHoxfTt/d27blZK+K2mupFeAI1NL5SxJC9M+10naPtU/QtKy3P4N66btX5H0jKTlkv5aUkjap+B3EPBN4IKI+H5EvBgR6yPizoj4bKpznqT/l9tnbHq/bdL6HZIulPQL4FVgmqTOuuP8naQ5aXk7SZdIekrSs5IukzR8K/85bBBwgrDB5EvAFGASsCfwAnBpbvtPgX2BtwMPAj+s2/9U4EJgR+DnqezjwNHAOOAA4NO9HL+wrqSjgTOBDwH7AEf08h7jgb2AWb3UKeOTwOlkv8tlwHhJ++a2nwpcnZYvBvYDDkzxjSJrsdgQ5wRhg8nngXMiYllEvAacB5xU+2YdEVdExEu5bX8saefc/j+KiF+kb+y/T2XfjojlEbES+DHZh2gjjep+HPi/EbE4Il5Nx25kt/TzmXK/ckNXpuOtjYgXgR8BpwCkRPFOYE5qsZwO/F1ErIyIl4B/Bk7eyuPbIOAEYYPJGOAmSaskrQIeBdYBu0tqk3Rxuvz0O+DJtM/I3P5PF7znb3LLrwI79HL8RnX3rHvvouPU/Db93KOXOmXUH+NqUoIgaz3MTsmqHXgr8EDuvP0sldsQ5wRhg8nTwDERMSL32j4iesg+FI8nu8yzMzA27aPc/lUNbfwMWWdzzV691O0m+z1O7KXOK2Qf6jV/UFCn/ne5BWiXdCBZoqhdXnoeWA28O3fOdo6I3hKhDRFOEDZQDZO0fe61Ddm19gsljQGQ1C7p+FR/R+A1sm/obyW7jNJfrgdOk/QuSW8FvtqoYmTj758JfFXSaZJ2Sp3vH5B0ear2EPBBSXunS2TTNxdARLxOdmfUDGBXsoRBRKwHvgf8q6S3A0gaJWnym/1lbfBwgrCBai7ZN9/a6zzgW8Ac4GZJLwH3Aoek+jOBXwM9wCNpW7+IiJ8C3wZuB5bkjv1ag/qzgE8AnwGWA88C/4usH4GIuAW4DlgIPAD8pGQoV5O1oG6IiLW58n+oxZUuv/0XWWe5DXHyhEFm/UvSu4CHge3qPqjNWopbEGb9QNIJ6XmDXYCvAz92crBW5wRh1j8+BzwHPE52Z9XfNDccs83zJSYzMyvkFoSZmRXaptkB9JWRI0fG2LFjmx2GmdmA8sADDzwfEYUPRg6aBDF27Fg6Ozs3X9HMzDaQ9OtG23yJyczMCjlBmJlZIScIMzMr5ARhZmaFKksQkq6Q9Jykhxtsl6RvS1qSZuE6KLftU5J+lV6fqipGMzNrrMq7mK4EvkM2SFqRY8hm99qXbEC17wKHpCkivwZ0kA1Z/ICkORHxQoWxbjC7q4cZ87pZvmo1e44YzrTJ45kycdRW1zUz62tVfwZVliAi4i5JY3upcjwwMw1vfK+kEZL2IJuO8ZY0KxeSbiGbxvGaqmKtneSeVasRbwyk37NqNdNvXASwyUmf3dXD9BsXsfr1dZuta2bW1/rjM6iZz0GMYuNZr5alskblm5B0Otl0iey9995vKoj6k1w/8Mjq19fxlVkLueb+pzYq73pqFWvWrS9V18ysrzX6DJoxr7vPEsSA7qSOiMsjoiMiOtrb39wMiTPmdW9IDo3U/yM0Kuut3MysLzX6rFm+anWfHaOZLYgeNp56cXQq6yG7zJQvv6OqIMqczFEjhnPd5w7dqOywi2+jp2DforpmZn2t0WfQniOG99kxmtmCmAP8Zbqb6X3AixHxDDAP+FNJu6Sx8/80lVVicydz+LA2pk3edHKtaZPHM3xYW6m6ZmZ9rT8+gyprQUi6hqwlMFLSMrI7k4YBRMRlZFNGHks21eGrwGlp20pJFwDz01udX+uwrsK0yeM36oMANnRUj+rlroBame9iMrNm6I/PoEEzH0RHR0e82cH6Znf18JVZC1mzbn2vScHMbLCR9EBEdBRtGzSjuW6NKRNHbbjzyP0HZmaZAX0XU1+Y3dXDYRffxn1PrKTrqVXM7uppdkhmZi1hSLcg6p+BWLNuvR92MzNLhnQLougZiNqDJmZmQ92QThCNnoHoywdNzMwGqiGdIBo9A9GXD5qYmQ1UQzpB+GE3M7PGhnQnda0j2s9AmJltaki3IMzMrLEhnSBqt7nWRkWsjafuZyHMzIZ4gvBtrmZmjQ3pBOHbXM3MGhvSCcK3uZqZNTakE4RvczUza8y3ueLbXM3MigzpBAEe6tvMrJEhfYnJzMwac4IwM7NCThBmZlbICcLMzAo5QZiZWSEnCDMzK+QEYWZmhSpNEJKOltQtaYmkswu2j5F0q6SFku6QNDq37euSHk6vT1QZp5mZbaqyBCGpDbgUOAaYAJwiaUJdtUuAmRFxAHA+cFHa98+Ag4ADgUOAsyTtVFWsZma2qSpbEAcDSyJiaUSsAa4Fjq+rMwG4LS3fnts+AbgrItZGxCvAQuDoCmM1M7M6VSaIUcDTufVlqSxvATA1LZ8A7Chpt1R+tKS3ShoJHAnsVX8ASadL6pTUuWLFij7/BczMhrJmd1KfBUyS1AVMAnqAdRFxMzAXuBu4BrgHWFe/c0RcHhEdEdHR3t7ej2GbmQ1+VSaIHjb+1j86lW0QEcsjYmpETATOSWWr0s8LI+LAiPgwIOCxCmM1M7M6VSaI+cC+ksZJ2hY4GZiTryBppKRaDNOBK1J5W7rUhKQDgAOAmyuM1czM6lQ23HdErJV0BjAPaAOuiIjFks4HOiNiDnAEcJGkAO4Cvph2Hwb8tySA3wF/ERFrq4rVzMw2Vel8EBExl6wvIV92bm55FjCrYL/fk93JZGZmTdLsTmozM2tRThBmZlbICcLMzAo5QZiZWSEnCDMzK+QEYWZmhZwgzMyskBOEmZkVcoIwM7NCThBmZlbICcLMzAo5QZiZWSEnCDMzK+QEYWZmhTabIGoT95iZ2dBSpgVxr6QbJB2rNIPPYDK7q4eup1Zx3xMrOezi25jd1bP5nczMhoAyCWI/4HLgk8CvJP2zpP2qDat/zO7qYfqNi1izbj0APatWM/3GRU4SZmaUSBCRuSUiTgE+C3wKuF/SnZIOrTzCCs2Y183q19dtVLb69XXMmNfdpIjMzFrHZqccTX0Qf0HWgngW+BIwBzgQuAEYV2F8lVq+avUWlZuZDSVl5qS+B/gBMCUiluXKOyVdVk1Y/WPPEcPpKUgGe44Y3oRozMxaS5k+iPERcUFdcgAgIr5eQUz9Ztrk8Qwf1rZR2fBhbUybPL5JEZmZtY4yCeJmSSNqK5J2kTSvupD6z5SJo7ho6v5s25adhlEjhnPR1P2ZMnFUkyMzM2u+MpeY2iNiVW0lIl6Q9PbqQupfUyaO4pr7nwLgus8N6D53M7M+VaYFsU7S3rUVSWOAKPPmko6W1C1piaSzC7aPkXSrpIWS7pA0OrftXyQtlvSopG8PxmcwzMxaWZkWxDnAzyXdCQg4HDh9cztJagMuBT4MLAPmS5oTEY/kql0CzIyIqyQdBVwEfFLS+4HDgANSvZ8Dk4A7Sv1WZma21TabICLiZ5IOAt6Xir4cEc+XeO+DgSURsRRA0rXA8UA+QUwAzkzLtwOza4cFtge2JUtKw8husTUzs35SdrC+dcBzwO+ACZI+WGKfUcDTufVlqSxvATA1LZ8A7Chpt4i4hyxhPJNe8yLi0foDSDpdUqekzhUrVpT8VczMrIwyg/X9NXAXMA/4p/TzvD46/lnAJEldZJeQesj6PPYB3gWMJksqR0k6vH7niLg8IjoioqO9vb2PQjIzMyjXgvhb4L3AryPiSGAisKrEfj3AXrn10alsg4hYHhFTI2IiWV8H6Y6pE4B7I+LliHgZ+CngW4zMzPpRmQTx+4j4PYCk7SLil0CZJ8nmA/tKGidpW+BksiE6NpA0UlIthunAFWn5KbKWxTaShpG1Lja5xGRmZtUpkyCWpQflZgO3SPoR8OvN7RQRa4EzyC5JPQpcHxGLJZ0v6bhU7QigW9JjwO7Ahal8FvA4sIisn2JBRPy47C9lZmZbr8xdTCekxfMk3Q7sDPyszJtHxFxgbl3ZubnlWWTJoH6/dcDnyhzDzMyq0WuCSM8yLI6IdwJExJ39EpWZmTVdr5eY0jf57vyT1GZmNjSUeZJ6F2CxpPuBV2qFEXFc413MzGygK5Mgvlp5FGZm1nLKdFK738HMbAgqM+XoS7wxeuu2ZOMivRIRO1UZmJmZNVeZFsSOteU05PbxvDFwn5mZDVJlB+sDIDKzgcnVhGNmZq2izCWmqbnVtwAdwO8ri8jMzFpCmbuYPppbXgs8SXaZyczMBrEyfRCn9UcgZmbWWsrMB3FVGqyvtr6LpCt62cXMzAaBMp3UB6Q5GgCIiBfI5oQwM7NBrEyCeIukXWorknalXN+FmZkNYGU+6L8B3CPphrT+Md6Yt8HMzAapMp3UMyV1AkeloqkR8Ui1YZmZWbOV6aR+H/B0RHwnIr5DNsPcIdWH1j9md/XQ9dQq7ntiJYddfBuzu3o2v5OZ2RBQpg/iu8DLufWXU9mAN7urh+k3LmLNuvUA9KxazfQbFzlJmJlRLkEoImqD9RER6xkkndQz5nWz+vV1G5Wtfn0dM+Z1NykiM7PWUSZBLJX0PyQNS6+/BZZWHVh/WL5q9RaVm5kNJWUSxOeB9wM9wDLgEOCzVQbVX/YcMXyLys3MhpLNJoiIeC4iTo6It0fE7sBfAUdUHlk/mDZ5PMOHtW1UNnxYG9Mmj29SRGZmraPUcN+S2iQdK+kHwBPAJ6oNq39MmTiKi6buz7Zt2WkYNWI4F03dnykTRzU5MjOz5uu1s1nSJOBU4FjgfuAw4B0R8WqZN5d0NPAtoA34fkRcXLd9DHAF0A6sBP4iIpZJOhL411zVdwInp7ko+tSUiaO45v6nALjuc4f29dubmQ1YDVsQkpYBFwE/ByZExInA6i1IDm3ApcAxwATgFEkT6qpdAsyMiAOA89PxiIjbI+LAiDiQ7AG9V4Gbt+QXMzOzrdPbJaZZwJ5kl5M+KultvDE3dRkHA0siYmlErAGuZdN5JCYAt6Xl2wu2A5wE/LRsYjIzs77RMEFExJeBcWRjMR0BdAPtkj4uaYcS7z0KeDq3viyV5S0AajPWnQDsKGm3ujonA9cUHUDS6ZI6JXWuWLGiREhmZlZWr53UaQ7q2yPidLJkcQrZt/wn++j4ZwGTJHUBk8hupd3w5JqkPYD9gXkN4rs8IjoioqO9vb2PQjIzM9iCJ6Ij4nXgJ8BPJJV5UKAH2Cu3PjqV5d9zOakFkVolJ+bnngA+DtyUjm1mZv2o1G2u9SKizKPG84F9JY2TtC3ZpaI5+QqSRkqqxTCd7I6mvFNocHnJzMyq9aYSRBkRsRY4g+zy0KPA9RGxWNL5ko5L1Y4AuiU9BuxObp4JSWPJWiB3VhWjmZk1VumgexExF5hbV3ZubnkW2d1SRfs+yaad2mZm1k82myAk7QdMA8bk60fEUQ13MjOzAa9MC+IG4DLge+TuMDIzs8GtTIJYGxGDYoIgMzMrr0wn9Y8lfUHSHpJ2rb0qj8zMzJqqTAviU+nntFxZAO/o+3DMzKxVbDZBRMS4/gjEzMxaS5m7mIYBfwN8MBXdAfwfP91sZja4lbnE9F1gGPBvaf2TqeyvqwrKzMyar0yCeG9E/HFu/TZJC6oKyMzMWkOZu5jWSfrD2oqkd+DnIczMBr0yLYhpwO2SlgIie6L6tEqjMjOzpitzF9OtkvYFxqei7oh4rdqwzMys2RomCElHRcRtkqbWbdpHEhFxY8WxmZlZE/XWgphENl/0Rwu2BeAEYWY2iDVMEBHxtbR4fkQ8kd8myQ/PmZkNcmXuYvqPgrLCORzMzGzw6K0P4p3Au4Gd6/ohdgK2rzowMzNrrt76IMYDHwFGsHE/xEvAZyuMyczMWkBvfRA/An4k6dCIuKcfYzIzsxZQ5kG5LklfJLvctOHSUkR8prKozMys6cp0Uv8A+ANgMnAnMJrsMpOZmQ1iZRLEPhHxVeCViLgK+DPgkGrDMjOzZiuTIGrzPqyS9EfAzsDbqwvJzMxaQZkEcbmkXYCvAnOAR4B/KfPmko6W1C1piaSzC7aPkXSrpIWS7pA0Ordtb0k3S3pU0iOSxpb7lczMrC+UGazv+2nxTrZgHmpJbcClwIeBZcB8SXMi4pFctUuAmRFxlaSjgIvIJiQCmAlcGBG3SNoBWF/22GZmtvV6e1DuzN52jIhvbua9DwaWRMTS9H7XAseTtUBqJgC149wOzE51JwDbRMQt6Vgvb+ZYZmbWx3q7xLRjenWQzUk9Kr0+DxxU4r1HAU/n1pelsrwFQO0p7ROAHSXtBuxH1udxo6QuSTNSi2Qjkk6X1Cmpc8WKFSVCMjOzshomiIj4p4j4J7LbWg+KiL+PiL8H3gPs3UfHPwuYJKmLbPTYHrLZ6rYBDk/b30t2aevTBTFeHhEdEdHR3t7eRyGZmRmU66TeHViTW1+TyjanB9grtz46lW0QEcsjYmpETATOSWWryFobD0XE0ohYS3bpqUyrxczM+kiZJ6lnAvdLuimtTwGuLLHffGDfNDR4D3AycGq+gqSRwMqIWA9MB67I7TtCUntErACOAjpLHNPMzPrIZlsQEXEh2RzUL6TXaRFxUYn91gJnAPOAR4HrI2KxpPMlHZeqHQF0S3qMrFVyYdp3HdnlpVslLSKbC/t7W/i7mZnZVujtLqadIuJ3knYFnkyv2rZdI2Ll5t48IuYCc+vKzs0tz6LB3BLpDqYDNncMMzOrRm+XmK4mG+77AbIpRmuU1ks/E2FmZgNPb8N9fyT99PSiZmZDUG+XmHq9aygiHuz7cMzMrFX0donpG71sC7I7i8zMbJDq7RLTkf0ZiJmZtZYyz0GQhvmewMYzys2sKigzM2u+zSYISV8je15hAtktq8cAPyd7gM7MzAapMkNtnAT8CfCbiDgN+GOySYPMzGwQK5MgVqehMNZK2gl4jo3HWDIzs0GoTB9Ep6QRZENdPAC8DNxTZVBmZtZ8vT0HcSlwdUR8IRVdJulnwE4RsbBfojMzs6bprQXxGHCJpD2A64FrIqKrf8IyM7Nm623CoG9FxKFkE/n8FrhC0i8lfU3Sfv0WoZmZNUWZ4b5/HRFfT5P6nEI2H8SjVQdmZmbNtdkEIWkbSR+V9EPgp0A3b8wjbWZmg1RvndQfJmsxHAvcD1wLnB4Rr/RTbGZm1kS9dVJPJ5sT4u8j4oV+isfMzFpEb4P1ebRWM7MhrMyT1GZmNgQ5QZiZWSEnCDMzK+QEYWZmhZwgzMysUKUJQtLRkrolLZF0dsH2MZJulbRQ0h2SRue2rZP0UHrNqTJOMzPbVKkpR98MSW3ApcCHgWXAfElzIuKRXLVLgJkRcZWko4CLgE+mbasj4sCq4jMzs95V2YI4GFgSEUsjYg3Zk9jH19WZANyWlm8v2G5mZk1SZYIYBTydW1+WyvIW8Ma4TicAO0raLa1vL6lT0r2SphQdQNLpqU7nihUr+jB0MzNrdif1WcAkSV1kw4r3AOvStjER0QGcCvxvSX9Yv3NEXB4RHRHR0d7e3m9Bm5kNBZX1QZB92Ofnrh6dyjaIiOWkFoSkHYATI2JV2taTfi6VdAcwEXi8wnjNzCynyhbEfGBfSeMkbQucDGx0N5KkkZJqMUwHrkjlu0jarlYHOAzId26bmVnFKksQEbEWOAOYRzbB0PURsVjS+ZKOS9WOALolPQbsDlyYyt8FdEpaQNZ5fXHd3U9mZlaxKi8xERFzgbl1ZefmlmcBswr2uxvYv8rYzMysd83upDYzsxblBGFmZoWcIMzMrJAThJmZFXKCMDOzQk4QZmZWyAnCzMwKOUGYmVkhJwgzMyvkBGFmZoWcIMzMrJAThJmZFXKCMDOzQk4QZmZWyAnCzMwKOUGYmVkhJwgzMyvkBGFmZoWcIMzMrJAThJmZFXKCMDOzQk4QZmZWyAnCzMwKVZogJB0tqVvSEklnF2wfI+lWSQsl3SFpdN32nSQtk/SdKuM0M7NNVZYgJLUBlwLHABOAUyRNqKt2CTAzIg4Azgcuqtt+AXBXVTGamVljVbYgDgaWRMTSiFgDXAscX1dnAnBbWr49v13Se4DdgZsrjNHMzBqoMkGMAp7OrS9LZXkLgKlp+QRgR0m7SXoL8A3grN4OIOl0SZ2SOlesWNFHYZuZGTS/k/osYJKkLmAS0AOsA74AzI2IZb3tHBGXR0RHRHS0t7dXH62Z2RCyTYXv3QPslVsfnco2iIjlpBaEpB2AEyNilaRDgcMlfQHYAdhW0ssRsUlHt5mZVaPKBDEf2FfSOLLEcDJwar6CpJHAyohYD0wHrgCIiD/P1fk00OHkYGbWvyq7xBQRa4EzgHnAo8D1EbFY0vmSjkvVjgC6JT1G1iF9YVXxNDK7q4eup1Zx3xMrOezi25jd1bP5nczMhgBFRLNj6BMdHR3R2dm5RfvM7uph+o2LWP36ug1lw4e1cdHU/Zkysb4/3cxs8JH0QER0FG1rdid1U82Y171RcgBY/fo6ZszrblJEZmatY0gniOWrVm9RuZnZUDKkE8SeI4ZvUbmZ2VAypBPEtMnjGT6sbaOy4cPamDZ5fJMiMjNrHVXe5tryah3RM+Z1s3zVavYcMZxpk8e7g9rMjCGeICBLEk4IZmabGtKXmMzMrDEnCDMzK+QEYWZmhZwgzMyskBOEmZkVGjRjMUlaAfx6K95iJPB8H4VTNcdanYEUr2OtzkCKd2tjHRMRhRPqDJoEsbUkdTYasKrVONbqDKR4HWt1BlK8VcbqS0xmZlbICcLMzAo5Qbzh8mYHsAUca3UGUryOtToDKd7KYnUfhJmZFXILwszMCjlBmJlZoSGfICQdLalb0hJJZzc7njxJe0m6XdIjkhZL+ttUfp6kHkkPpdexzY61RtKTkhaluDpT2a6SbpH0q/RzlxaIc3zu/D0k6XeSvtxK51bSFZKek/RwrqzwXCrz7fR3vFDSQS0Q6wxJv0zx3CRpRCofK2l17hxf1gKxNvx3lzQ9ndduSZP7M9Ze4r0uF+uTkh5K5X17biNiyL6ANuBx4B3AtsACYEKz48rFtwdwUFreEXgMmACcB5zV7PgaxPwkMLKu7F+As9Py2cDXmx1nwd/Bb4AxrXRugQ8CBwEPb+5cAscCPwUEvA+4rwVi/VNgm7T89VysY/P1WuS8Fv67p/9vC4DtgHHp86Kt2fHWbf8GcG4V53aotyAOBpZExNKIWANcCxzf5Jg2iIhnIuLBtPwS8CgwECevOB64Ki1fBUxpXiiF/gR4PCK25kn8PhcRdwEr64obncvjgZmRuRcYIWmPfgmU4lgj4uaIWJtW7wVG91c8vWlwXhs5Hrg2Il6LiCeAJWSfG/2mt3glCfg4cE0Vxx7qCWIU8HRufRkt+gEsaSwwEbgvFZ2Rmu5XtMIlm5wAbpb0gKTTU9nuEfFMWv4NsHtzQmvoZDb+D9aq5xYan8tW/1v+DFkLp2acpC5Jd0o6vFlB1Sn6d2/183o48GxE/CpX1mfndqgniAFB0g7AfwBfjojfAd8F/hA4EHiGrInZKj4QEQcBxwBflPTB/MbI2sEtc2+1pG2B44AbUlErn9uNtNq5bETSOcBa4Iep6Blg74iYCJwJXC1pp2bFlwyYf/c6p7Dxl5s+PbdDPUH0AHvl1kenspYhaRhZcvhhRNwIEBHPRsS6iFgPfI9+bvL2JiJ60s/ngJvIYnu2drkj/XyueRFu4hjgwYh4Flr73CaNzmVL/i1L+jTwEeDPU0IjXa75bVp+gOy6/n5NC5Je/91b8rwCSNoGmApcVyvr63M71BPEfGBfSePSN8mTgTlNjmmDdH3x34FHI+KbufL8teUTgIfr920GSW+TtGNtmayT8mGyc/qpVO1TwI+aE2Ghjb6Bteq5zWl0LucAf5nuZnof8GLuUlRTSDoa+ApwXES8mitvl9SWlt8B7AssbU6UG2Jq9O8+BzhZ0naSxpHFen9/x9fAh4BfRsSyWkGfn9v+7I1vxRfZ3R+PkWXac5odT11sHyC7hLAQeCi9jgV+ACxK5XOAPZoda4r3HWR3fCwAFtfOJ7AbcCvwK+C/gF2bHWuK623Ab4Gdc2Utc27JEtczwOtk177/qtG5JLt76dL0d7wI6GiBWJeQXb+v/e1eluqemP4+HgIeBD7aArE2/HcHzknntRs4phX+DlL5lcDn6+r26bn1UBtmZlZoqF9iMjOzBpwgzMyskBOEmZkVcoIwM7NCThBmZlbICcIGBEn/KunLufV5kr6fW/+GpDN72f9KSSel5TskbTLJu6Rhki5OI6U+KOkeScekbU9KGvkm4t5w3AbbL02jbj5SNwrnSZLm1kZA7UuS9pD0k162byvprvQglg1hThA2UPwCeD+ApLcAI4F357a/H7h7K49xAdkIun8U2XAhU8hG0a1MRHwxIg4ke77l8Yg4ML1mRcSxEbGqgsOeSfa0cKOY1pA9a/GJCo5tA4gThA0UdwOHpuV3kz3p+pKkXSRtB7wLeFDSuZLmS3pY0uXpafTNkvRW4LPAlyLiNdgw/ML1BXXPTO//cF2r5i/TYG8LJP2gYL8LUouirWRMT0oamcb4/2Xa9zFJP5T0IUm/SK2dg1P9t6WB5u5Pg7U1Gpn4ROBnaZ93p/oPpdj3TXVmA39eJk4bvNyEtAEhIpZLWitpb7LWwj1ko2oeCrwILIqINZK+ExHnA6QP6Y8APy5xiH2ApyIbDLEhSe8BTgMOIXt6+T5JdwJrgH8E3h8Rz0vatW6/GWStkdPizT2dug/wMbJRUecDp5I9aX8c8D/JWjvnALdFxGfSpan7Jf1XRLySi2Mc8EItCQKfB74VET9Mw83UktfDwHvfRJw2iLgFYQPJ3WTJoZYg7smt/yLVOVLSfZIWAUex8WWovvAB4KaIeCUiXgZuJBty+Sjghoh4HiAi8uP3f5VsOI/Pv8nkAPBERCyKbDC5xcCt6b0WkU0SA9nYV2crm13sDmB7YO+699kDWJFbvwf4n5L+ARgTEatT/OuANbWxtWxocoKwgaTWD7E/2Tfce8laEO8H7pa0PfBvwEkRsT/ZdfbtS773EmBvVTPs9HzgPfWtii30Wm55fW59PW9cCRBwYq4fY++IeLTufVaTOycRcTVZK2Q1MFfSUbm62wG/34qYbYBzgrCB5G6yS0YrIxuaeSUwgixJ3M0bH3zPK5tDo+HdQ/UiG23034FvpUsttZExP1ZX9b+BKZLemkasPSGV3QZ8TNJuad98MvgZcDHwnxV/I58HfKnW7yJpYkGdx3ijxVEb8XNpRHybbGTYA1L5bsDzEfF6hfFai3OCsIFkEdndS/fWlb0YEc+nO36+R9a6mEf2zX1L/CPZ5ZdHlE0Q/xNgoz6JyKaAvZJsyOf7gO9HRFdELAYuBO6UtAD4Zt1+N6TY5kgavoVxlXUBMAxYKGlxWt9I6o94XNI+qejjwMPpstQfATNT+ZHAf1YUpw0QHs3VbIiRdALwnoj4x17q3AicHRGP9V9k1mp8F5PZEBMRN9UuhRVJl9hmOzmYWxBmZlbIfRBmZlbICcLMzAo5QZiZWSEnCDMzK+QEYWZmhf4/G6ItbVv0SfoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "plt.title('Learning Curve')\n",
    "plt.xlabel('Wall Clock Time (s)')\n",
    "plt.ylabel('Validation Accuracy')\n",
    "plt.scatter(time_history, 1 - np.array(valid_loss_history))\n",
    "plt.step(time_history, 1 - np.array(best_valid_loss_history), where='post')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f74e119f1c0>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABM+ElEQVR4nO3dd3hVVdbA4d9KgQQILSQQaggdIQQIvXdEFKVIUREVkGobxzY6tmFGHfVTUFSwYQFRqiCiSO8dQocAAUINLSSQhJT9/XFuMgFCchNy701Z7/PcJ8k5+5y7jsS7cs7ee20xxqCUUkoBuLk6AKWUUnmHJgWllFJpNCkopZRKo0lBKaVUGk0KSiml0ni4OoA7Ua5cORMYGOjqMJRSKl/ZunXreWOMX0b78nVSCAwMZMuWLa4OQyml8hUROXa7ffr4SCmlVBpNCkoppdI4LCmIiJeIbBKRnSKyR0TetG1fLSI7bK9TIjLvpuOaiUiSiPR3VGxKKaUy5sg+hQSgszEmVkQ8gTUi8rsxpl1qAxGZDcxP97M78C7wpwPjUkplQ2JiIpGRkcTHx7s6FJVNXl5eVK5cGU9PT7uPcVhSMFZRpVjbj562V1qhJREpCXQGHkt32HhgNtDMUXEppbInMjISHx8fAgMDERFXh6PsZIzhwoULREZGUr16dbuPc2ifgoi4i8gO4BywxBizMd3u+4GlxpgrtraVgAeAz7I450gR2SIiW6KiohwTuFIqTXx8PL6+vpoQ8hkRwdfXN9t3eA5NCsaYZGNMCFAZaC4iDdLtHgzMSPfzR8CLxpiULM45xRgTaowJ9fPLcJitUiqXaULIn3Ly7+aUeQrGmMsishzoCewWkXJAc6w7g1ShwE+2iygH9BKRJGPMPGfEqPK4pATYMw/8akPFxq6ORqkCy5Gjj/xEpLTte2+gG7Dftrs/sNAYk3ZfY4ypbowJNMYEArOAMZoQFAAH/4TJLWHuSJjSCeaPg6vnXR2VcrJ58+YhIuzfvz/Lth999BHXrl3L8Xt9++23jBs3zu7td8IR57wTjnx8FAAsF5EwYDNWn8JC275B3PjoSKlbXTwK0wfB9AEg7jBoBrQeBztnwMQmsOFzSE5ydZTKSWbMmEHbtm2ZMSPrj447TQqFmcOSgjEmzBjT2BgTbIxpYIx5K92+jsaYxZkcO8wYM8tRsak87vo1WDYBPm0BEauh21sweh3U7QXd/2V9X6kJLH4RvmgPEWtcHbFysNjYWNasWcNXX33FTz/9lLY9OTmZ559/ngYNGhAcHMykSZOYOHEip06dolOnTnTq1AmAEiVKpB0za9Yshg0bBsCCBQto0aIFjRs3pmvXrpw9e9bumKKioujXrx/NmjWjWbNmrF27lpSUFAIDA7l8+XJau1q1anH27NkM2+dF+br2kSpgjIF9C+CPVyD6BDQcYCWEkhW5ePU6j3+xlhZBZXmhR13cH5kL+xfC4lfg23vgrr5WwihVydVXUaC9uWAPe09dydVz1q9YktfvvSvTNvPnz6dnz57Url0bX19ftm7dStOmTZkyZQoRERHs2LEDDw8PLl68SNmyZfnwww9Zvnw55cqVy/S8bdu2ZcOGDYgIX375Je+99x4ffPCBXXE//fTTPPvss7Rt25bjx4/To0cP9u3bR58+fZg7dy6PPfYYGzdupFq1apQvX54hQ4Zk2D6v0aSg8oaog/D7C3BkOfjfBcMWQWAbABKTUxjz41bCIi+z48RljkZd5aNBIRSrdy/U7AprP4Y1/wcHF0P756HVOPAo6uILUrlpxowZPP300wAMGjSIGTNm0LRpU/766y9GjRqFh4f1UVa2bNlsnTcyMpKBAwdy+vRprl+/nq3x/H/99Rd79+5N+/nKlSvExsYycOBA3nrrLR577DF++uknBg4cmGn7vEaTgnKthBhY+R5smAyexeHu9yD0CXD/36/mvxbuZcORi3wwoBFX4hN5e+FeBk3ZwJdDQ/Ev6Q0dX4JGg607jKVvwfYfoOc7ULuHCy+sYMrqL3pHuHjxIsuWLWPXrl2ICMnJyYgI//3vf+0+R/qhmenH7Y8fP57nnnuO++67jxUrVvDGG2/Yfc6UlBQ2bNiAl5fXDdtbtWpFeHg4UVFRzJs3j1dffTXT9nmNFsRTrmEMhP0Ck0Jh3UTrQ338Vmjx5A0J4adNx5m2/hjD21anX9PKPNamOlMeCSX8XCz3f7qW/WdsjzLKVINBP8Ijc8HNA6Y/CD8+CBcOu+gCVW6ZNWsWjzzyCMeOHSMiIoITJ05QvXp1Vq9eTbdu3fjiiy9ISrIGHFy8eBEAHx8fYmJi0s5Rvnx59u3bR0pKCnPnzk3bHh0dTaVK1iPHadOmZSuu7t27M2nSpLSfd+zYAVgJ6IEHHuC5556jXr16+Pr6Zto+r9GkoJzvzG74phfMGQ4lA2D4UujzCZS4cTLiloiLvDZ/N+1qleOlu+umbe9avzw/P9mKZGPo/9l6Vh5MN7O9RmcYtdbqXzi2zhrK+tebcP2qs65O5bIZM2bwwAMP3LCtX79+zJgxg+HDh1O1alWCg4Np1KgR06dPB2DkyJH07NkzraP5nXfeoXfv3rRu3ZqAgIC087zxxhsMGDCApk2bZtn/cLOJEyeyZcsWgoODqV+/Pp9//nnavoEDB/LDDz+kPTrKqn1eIlaJovwpNDTU6CI7+UjcJVj+b9j8JXiVhq5vQONHwO3Wv01OXY7jvk/WUqKoO/PHtqVUsVsLep2OjuPxb7dw8GwMb/dpwJAWVW9sEHMGlrwOYT9ByUrQ/W2rQ1pn52bLvn37qFevnqvDUDmU0b+fiGw1xoRm1F7vFJTjpaTAtu9gUlMrIYQ+YT0qavpohgkhPjGZJ7/fStz1JKYODc0wIQAElPLml1GtaF+rHK/M3cV/Fu0jJSXdHzk+FaDvF/D4n1DMF2Y9DtPuhbN7HHWlSuV7mhSUY53cCl92gV/HQ7na8OQquOd9KJbxKBFjDC/NDmPXyWg+GtSYWuV9Mj19iaIeTB0ayiMtq/HFqiOMnb6NuOvJNzaq2gJGroDe/wdnd8Pn7WDRCxB3OXeuUakCRJOCcoyr561EMLULXDkJD0yBx36HCg0zPWzKqiPM23GKv3WrTbf65e16Kw93N97qcxev9a7P4j1nGDR1A1ExCTc2cnOH0Mdh/DYIfQw2T4VJTWDrNOtORikFaFJQuS05CTbZPnB3TIdWY2HcFmg0MMtn+SsOnOOdxfu5p2EA4zrXzNbbighPtK3OFw835eCZGO7/dC2Hzsbc2rBYWbjnAxi50rpzWfCUdScTuTVb76dUQaVJQeWeY+tgSgdY9DwEhFjlKHpMAK+SWR56JCqW8TO2U6e8D/8dEJzjUs3d76rAzCdbcj05hb6frWPNodsUzgsItu5c+k6FK6fgy84wbyzEnsvR+ypVUGhSUHfuymmYPQK+udt6Tv/gdzB0PvjVse/w+ERGfLcFDzdh6tBQihW5szmVwZVLM29sGyqW8mbYN5uYufl4xg1FIPhBGL8F2jwNYTOtzvANn0Fy4h3FoFR+pUlB5VxKMqydCJ+Ewt550P7vMG4z1O9j97DP5BTDMz/t4NiFa0x+qClVyhbLldAqlfZm1uhWtKrhy4uzd/Hu4v03jkxKr6iPVWNpzHqo3AwWv2R1Rh9dlSuxqDvn7u5OSEhI2isiIoIVK1bQu3fv2x4TEhLCoEGDbtg2bNgwihUrdsPEtmeeeQYR4fx5664yffG8jCQkJNC1a1dCQkKYOXPmHVxV7vn3v/+da+fSpKBybtm/YMlrENgWxmyAzq9Ckex9qH/w5wGW7T/H6/fWp1UN31wNz8fLk6+HNWNIi6p8tuIw42dsJz4x+fYHlKsFD8+GQdMh8Zo1fHXxy7kak8oZb29vduzYkfYKDAzMtP2+fftITk5m9erVXL1648TFmjVrMn/+fMAqPbFs2bK0Wc322L59O2DNSE4/OS0zycmZ/N7lAk0KyvWOrLSK0DV+BIbMBN8a2T7Fgp2nmLziMIObV+HhltUcECR4ursx4f4GvNKrLot2n2bw1A2cj024/QEiUPceGLsRmg6zajIdWemQ2JTjzJgxg0ceeYTu3bunJYBUgwYNSvsLf8WKFbRp0yatoF5Wzp07x8MPP8zmzZsJCQnh8OHDLF26lMaNG9OwYUMef/xxEhKs36/AwEBefPFFmjRpwi+//MKff/5Jq1ataNKkCQMGDEgrhrd582Zat25No0aNaN68OTExMURERNCuXTuaNGlCkyZNWLduHQCnT5+mffv2hISE0KBBA1avXs1LL71EXFwcISEhPPTQQ3f8304L4qnsu3oB5oy0/rK++90cnWL3yWj+PmsnodXK8OZ9DRy6BrCIMLJ9DaqWLcYzM3fwwOS1fDOsGTX9M5kD4eltFdU7shJ+e87qNNfKq/D7S3BmV+6es0JDuPudTJukfugBVK9e/Yb6RRmZOXMmS5YsYf/+/UyaNIkhQ4ak7atduza//vorly5dYsaMGTz88MP8/vvvdoXq7+/Pl19+yfvvv8/ChQuJj4+nY8eOLF26lNq1azN06FA+++wznnnmGQB8fX3Ztm0b58+fp2/fvvz1118UL16cd999lw8//JCXXnqJgQMHMnPmTJo1a8aVK1fw9vbG39+fJUuW4OXlxaFDhxg8eDBbtmxh+vTp9OjRg3/84x8kJydz7do12rVrxyeffJJrtZT0TkFljzEwfyzEXYR+X0GR4tk+xfnYBJ78fitlihXhs4ebUsTDOb+GPRsE8NPIVsRdT+aByetYF57Fkp6e3tZEuwvhVnlu5TLpHx9llRC2bNlCuXLlqFq1Kl26dGH79u1phfJS9e3bl59++omNGzfSrl27HMd14MABqlevTu3atQF49NFHWbXqf31RqY+XNmzYwN69e2nTpg0hISFMmzaNY8eOceDAAQICAmjWrBkAJUuWxMPDg8TEREaMGEHDhg0ZMGBAWsntZs2a8c033/DGG2+wa9cufHwyn9yZE3qnoLJn85dw8Hfo8R9rWGc2XU9KYcwP2zgfm8CsUa3x83HuX98hVUozd0wbHv92M0O/3sS/+zbkwdAqtz+gZle46wFY9T406Jejx2QFShZ/0ecFM2bMYP/+/Wn9DleuXGH27NmMGDEirc3AgQNp2rQpjz76KG4ZlFrJLcWLW380GWPo1q3bLUuJ7tqV8V3X//3f/1G+fHl27txJSkpKWrnt9u3bs2rVKn777TeGDRvGc889x9ChQ3M1Zr1TUPY7uwf++AfU6g4tR+foFG8u2MOmiIu81z+YhpVL5XKA9qlSthizRremZZAvL8wK4/0/Dtx+ZBJYCdC9iDX/Ih8XkCwMUlJS+Pnnn9m1axcRERFEREQwf/78Wz6Mq1WrxoQJExgzZswdvV+dOnWIiIggPDwcgO+//54OHTrc0q5ly5asXbs2rd3Vq1c5ePAgderU4fTp02zevBmAmJgYkpKSiI6OJiAgADc3N77//vu0jupjx45Rvnx5RowYwfDhw9m2bRsAnp6eJCbmzjBqTQrKPtevWQXlvEpBn8k5qjT6w4Zj/LjxOE92CKJPiGuXzSzl7ck3jzVjULMqfLI8nKdn7rj9yKSSAdDlNTi8DPZk/uhCOdfSpUupXLly2mv16tVUqlSJihUrprVp3749e/fu5fTp0zcc++STT1Kjxq13fteuXbvhnB9++OFt39/Ly4tvvvmGAQMG0LBhQ9zc3Bg1atQt7fz8/Pj2228ZPHgwwcHBtGrViv3791OkSBFmzpzJ+PHjadSoEd26dSM+Pp4xY8Ywbdo0GjVqxP79+9PuOFasWEGjRo1o3LgxM2fOTFuNbuTIkQQHB+dKR7OWzlb2WfgsbPkaHp4DNbtk+/BNRy8yZOoG2tYqx1ePNsPdLW+UrzbG8MWqI7zz+36aVivDlEea4lsig0daKckwtbNVjnvcJis5FhJaOjt/09LZKvftW2AlhNbjc5QQTl6OY/QPW6lathgfD2qcZxICWCOTRnWoweSHmrD7ZDR9P1vH4agM1s11c7eqrMaehWUTnB+oUk6iSUFlLvqkVe00IAQ6/zPbh8ddT2bkd1u4npTClKGhlPLOeG0EV+vVMIAZI1sSG59E38nr2HDkwq2NKjWBZsOtCquntjs/SKWcQJOCur2UZGs+QtJ16P81eBTJ1uHGGP4+ayd7T1/h48Eh1PTPvHyAqzWpWoZ5Y9tQrkQRHvlqI3O2Rd7aqMtrUNzPepyW4thZqnlJfn7MXJjl5N/NYUlBRLxEZJOI7BSRPSLypm37ahHZYXudEpF5tu19RCTMtn2LiLR1VGzKTqs/hGNrrLH6ORiK+dnKwywMO83fe9Shc1371kZwtSplizFndBuaBZbluZ93MnXVkRsbeJWCHv+27hS2fO2aIJ3My8uLCxcuaGLIZ4wxXLhwIW04q70c1tEs1hTV4saYWBHxBNYATxtjNqRrMxuYb4z5TkRKAFeNMUZEgoGfjTF1Mz67RTuaHejEJvi6pzVGv9+X2R5ttGz/WZ6YtoXewRWZOCjEoTOWHeF6UgpjftzG6kNRrHmx843zKYyB7++Hk9usAoA+FVwWpzMkJiYSGRlJfHy8q0NR2eTl5UXlypXx9LzxsW1mHc0Om7xmrGyT2mPnaXulZSARKQl0Bh6ztU/fu1c8fVvlZPHRMPsJKFUJen+Y7YQQfi6Gp2fsoH5ASd7rl/O1EVypiIcbr/SqS5cPz/L12qO82DPd3ycicM+HMLmVNW+j/1euC9QJPD09qV69uqvDUE7i0D4FEXEXkR3AOWCJMWZjut33A0uNMVfStX9ARPYDvwGP3+acI22Pl7ZERUU5LvjCyhhY8IzVwdzv62wPvYyOS2TEd1sp6unGlKGheBdxd0ycThDkV4JeDQL4Yf0xrsTfNDHItwa0ew52z7LmLyhVQDg0KRhjko0xIUBloLmINEi3ezAw46b2c22PjO4H3r7NOacYY0KNMaF+fn6OCbww2/Ej7JkDnV6BKs2ydWhyiuGpGduJvHSNzx5uSqXS3g4K0nlGd6xBTEIS368/duvONs9A2Rrw298gUR+tqILBKaOPjDGXgeVATwARKQc0x7ojyKj9KiDI1k45y/lwWPQCBLaDts9m+/D3/tjPyoNRvHlfA5oFlnVAgM7XoFIpOtT24+s1R4m7ftNoI08va73ni0esMuJKFQCOHH3kJyKlbd97A92A/bbd/YGFxpj4dO1r2jqnEZEmQFEgg8HiyiGSEmD249aw075TrMla2TB/x0m+WHmEh1tWZUiLqg4K0jXGdKzBhavX+XnLiVt31ugEDfrDmg/hwmHnB6dULnPknUIAsFxEwoDNWH0KC237BnHToyOgH7Db1gfxKTDQ6Bg451n6FpzeCX0+hZIVs26fTljkZV6YFUbz6mX5Z++7HBSg6zSvXtYqgbHqCInJKbc26PFv8PC21l3QX1mVzzksKRhjwowxjY0xwcaYBsaYt9Lt62iMWXxT+3eNMXcZY0KMMa2MMWscFZu6SfhfsP4Ta7Zu3Xuydei5mHie/H4r5UoU5bOHmjhtbQRnEhHGdKzByctx/Lrj1K0NfMpbk9qOrIDds50en1K5qeD9H6yyJ/YczB0F/vWh+7+ydWhCUjKjf9jGpWvXmTL0NoXkCojOdf2pW8GHz1YezrjMdujjULGxtaZz3GWnx6dUbtGkUJilpMC80ZAQY62i5mn/aKHzsQk8/OVGth67xPsDGnFXxYJdNVREGN2xBuHnYvlz79lbG6QWzLt2HpZlL7kqlZdoUijMNn5mPTrqMQHK17f7sP1nrtDnk7WERUYzaXBjegdnrw8iv7qnYQBVyxbjsxXhGZd8qNgYmo+0Vqc7udX5ASqVCzQpFFandsCS16Fubwh9wu7Dluw9S7/J60hKSeGXUa24t1HhSAgAHu5ujOpQg52R0aw7fJuBcZ3+ASXKWxMAk5OcGp9SuUGTQmGUEGuVsSjuB/dNsquMhTGGySvCGfn9Fmr6l+DXcW0Jrlza8bHmMf2aVsLfpyifLg/PuIFXSej5HzgTZt0xKJXPaFIojBa/aI2p7/sFFMt6kll8YjLP/byT9xYfoHdwRWY+2YryJbNXebGgKOrhzvB21Vl3+AI7TlzOuNFdD0CNLlbfwpXTGbdRKo/SpFDY7J4D23+Adn+D6u2zbH4uJp7BUzcwd/tJ/tatNhMHheDlmX/rGeWGIS2qUcrbk8m3u1sQscqNJ1+HP152bnBK3SFNCoXJpWPWs+7KzaDjS1k2330ymj6frGX/6Rg+f7gJ47vUypcVT3NbiaIePNo6kD/3nuXQ2ZiMG5UNgvZ/hz1z4dBfzg1QqTugSaGwSE6COSMAY62P4J75spi/7zrNgM/XI8Cs0a3o2SDAKWHmF4+1DsTb053PVmRS2qLNU+BbCxb9DRLjnBecUndAk0JhsfJdOLHRGktfJvC2zYwxTFx6iNE/bqNugA/zxrUp8HMQcqJM8SIMbl6V+TtPceLitYwbeRS1CuZdirBWsVMqH9CkUBhErIXV70PIQ9Cw/22bxScmM37Gdj5ccpC+jSsxY0RL/H0KZ4eyPUa0r46bwNTVR27fKKgDBA+0qqhGHXRecErlkCaFgu7aReuxUZnqcPd7t212JjqeB79Yz2+7TvPS3XX54MFGhb5DOSsBpbzp27gyMzefICom4fYNu/8LihTTgnkqX9CkUJAZAwuesuob9f8KipbIsNnOE5e575M1HD4Xy5RHQhnVoYZ2KNvpyQ5BXE9O4eu1R2/fqIQ/dHkdIlbDrl+cF5xSOaBJoSDb+g3sWwBd/mmVYMjArztP8eAX6yni4cbsMa3pVr+8k4PM3zJdsjO9po9BpVD44xWIu+S8AJXKJk0KBdW5/bD4FajRGVqNu2V3Sorhgz8P8NSM7TSqXJr5Y9tQt0JJFwSa/2W6ZGcqNzdbwbwL1toVSuVRmhQKosR4mPU4FCkO939ufSClc+16EmN+3MakZeE8GFqZH4a3KNBlrx0t0yU70wsIhhajYcs3ELnFeQEqlQ2aFAqiJf+Ec3vggc+tBWDSOXU5jv6frefPvWd49Z56vNsvuEAujONsmS7ZmV6nl8EnABY+owXzVJ6knwYFzYHfYdMX0HIM1Op2w66txy5x3ydrOXHxGl8Na8bwdkHaoZxLmlcvS2hmS3amKuoDd78DZ3bBpinOC1ApO2lSKEiunIZ5Y6BCQ+j6xg275myLZPCUDRQv6s7csa3pVMffNTEWUCLCmE7Wkp3zM1qyM71690Gt7rB8AkSfdE6AStlJk0JBkZIMc0dCUjz0+9qaTYvVofzO7/t57uedNK1Whnlj2lDT38fFwRZMnepYS3Z+frslO1OJQK//QkoSLM66BpVSzqRJoaBY/QEcXWV92PjVBiA2IYmR32/h85WHeahFVb57ojllihdxcaAFV5ZLdqZXJtAqmLfvVzj4p1PiU8oemhQKgoi1sOI/0PBBq5QFcOLiNfp/to7lB6J4q89d/Ov+Bni66z+3o93TMIBqvpks2Zle66egXB1Y9Dxcv039JKWcTD8l8rurF2D2cKuMRe8PQYTNERfp8+laTl2O49vHmjG0VaB2KDuJh7sbT7a3luxcG36bJTvTGhex/s0uH7NqUymVB2hSyM+MgXmj4dp5GPAtFPXh580nGDJ1A6W9PZk3tg3tavm5OspCJ3XJzskrbrMIT3qBbaHREFg7EaIOOD44pbLgsKQgIl4isklEdorIHhF507Z9tYjssL1Oicg82/aHRCRMRHaJyDoRaeSo2AqM9Z/CoT+g+wRMhYa8t3g/L8wOo2WQL3PHtCHIL+NaR8qxinq4M6JdUOZLdqbX/W1rouFCLZinXM+RdwoJQGdjTCMgBOgpIi2NMe2MMSHGmBBgPTDH1v4o0MEY0xB4G9BB3Jk5uRX+egPq9iYldDivzd/N5BWHGdy8Kt8Ma0apYpkvoqMca3CLqpkv2Zle8XJWfapja+DwUscHp1QmHJYUjCXW9qOn7ZX2Z5CIlAQ6A/Ns7dcZY1IrhW0AKjsqtnwvPhp+eQx8AkjqPYnnZ4Xxw4bjPNkhiH8/0AAP7VB2ufRLdh683ZKd6TV+xJrpvO4TxwenVCYc+ukhIu4isgM4BywxxmxMt/t+YKkx5koGhz4B/H6bc44UkS0isiUqKiq3Q877jIFfn4LoSK4/MIWxc48wZ/tJ/t6jDi/1rKsdynlI6pKdn2e2ZGcqjyLQ4kk4stya7ayUizg0KRhjkm2PiSoDzUWkQbrdg4EZNx8jIp2wksKLtznnFGNMqDEm1M+vEHaibv0G9s7jesd/8MRSN/7Yc5Y37q3P2E41NSHkMWWKF2FIiyyW7Eyv6TDwLK53C8qlnPKcwRhzGVgO9AQQkXJAc+C39O1EJBj4EuhjjMliPF8hdHYPLH6ZxOqdeWhPC9aGn+e//YMZ1qa6qyNTtzG8nbVk55RVmSzZmcq7DDQZCrtnafkL5TKOHH3kJyKlbd97A92A/bbd/YGFxpj4dO2rYnU6P2KM0cVsb3b9KvwyjJSiJXn00uPsOHmFT4Y0YUBoFVdHpjKRumTnz1uyWLIzVcvRYFKsooZKuYAj7xQCgOUiEgZsxupTWGjbN4hbHx39E/AFJtuGq2rB+fQW/R1z/hB/TxnP1vMeTBkaSq+GAa6OStlhVMcaJGa1ZGeqMtWg/v3WmgvxGXW3KeVYjhx9FGaMaWyMCTbGNDDGvJVuX0djzOKb2g83xpRJHa5qjAl1VGz5zs6ZsONHpnn0549rdfju8eZa5TQfqV6uOHc3tJbsjI7LZMnOVK3HQcIV2P6944NT6iY6djGvOx9OyoJn2C71mJTcj+kjWtAiyNfVUalsGt3BWrLzhw2ZLNmZqlJTqNYGNnymC/Eop9OkkJclxhM3YyjRSe686v4s059sS3Dl0q6OSuWA3Ut2pmo9HqJPwN55Do9NqfQ0KeRhZ2c9j/eFPUzwfIpPR/WmTgVdByE/G9uppn1LdgLU6gG+tWDdRC19oZxKk0IetWfpD5Q/8D0/e/bhb+PGE1iuuKtDUnfI7iU7AdzcoNVYOL0TItY4J0ClyGZSEJEytrkEyoFWbtxM5VUvcMC9Fp3HfUpAKW9Xh6Ryid1LdgI0GgTFysG6SY4PTCmbLJOCiKwQkZIiUhbYBkwVkQ8dH1rhNGfzUXx+G4WHmyFg+AzKldJHRgWJ3Ut2Anh6Q/ORViVcLautnMSeO4VStvpEfYHvjDEtgK6ODatw+m59BOfmv0YTt3Dc+3xCyYBarg5J5TIRYUynmvYt2QnQ7Anw8IL1WvpCOYc9ScFDRAKAB4GFWTVWOfPp8nCWLpjOKI8FJDUehldIP1eHpBykV4MKVPMtxmR7luwsXg5ChsDOnyDGjiSi1B2yJym8BfwBhBtjNotIEHDIsWEVHsYY3l28n2l/bOAT7y8w/vXx6PWOq8NSDpS6ZGeYPUt2ArQcC8mJsHmq44NThV6WScEY84ttVvIY289HjDH6Z2wuSEkx/HP+Hr5YcYgZ5b6ihNt1ZMC31rNkVaBla8nOcjWh7j2w+Uu4bke1VaXugMftdojIJNItinMzY8xTDomokEhKTuGFWWHM2X6S72qspMbJbdBnMvjVcXVoyglSl+ycsGgf249fonHVMpkf0Goc7F8IO36E5iOcE6QqlDK7U9gCbM3kpXIoISmZsdO3MWf7ST5sHkO7U19B8EDr2bEqNNKW7LRnEZ6qLaFSqLUud4odM6KVyqHb3ikYY6al/1lEihlj9N71Dl27nsST329l9aHz/KdHAH23/Q3KVId7PgBdJKdQKVHUg2GtA/l46SEOno2hdvlMhh+LWKUvfnkUDiyCevc6L1BVqNgzT6GViOzFthaCiDQSkckOj6wAuhKfyNCvNlmL4/RrwOBT78C1izDgWyiq8xEKo2GtAylWxM4lO+vdC6Wr6WQ25VD2jD76COgBXAAwxuwE2jswpgLpQmwCQ6ZuYGfkZWtxnMRfrUlJPSZAgE4SL6zKFC/C4OZ2Ltnp5m6VvjixEY5vzLytUjlkV5kLY8zNFbz0oWY2RMclMmjKBg6djbUWxylzCv56A+r2hmbDXR2ecrFsLdkZ8hB4lYb1eregHMOepHBCRFoDRkQ8ReR5YJ+D4yowjDG8OCuMo+ev8s1jzehUrSjMegx8KkKfT7QfQRFQypt+TSoz054lO4uWsGY571sIF+1IIkplkz1JYRQwFqgEnARCbD8rO0xbF8HiPWd4sWddWgf5wq/j4cpJ6P+1tVC7UsDI9kEkJqfw3fqIrBs3HwnunrBeu/ZU7rMnKYgx5iFjTHljjL8x5mFjjB3TMFVY5GUmLNpHl7r+DG9XHbZ8DXvnQ+fXoEozV4en8pAgvxJ0r1+e79Yf42pCFqut+VSAhg/C9h+sgQpK5SJ7ksJaEflTRJ4QkdKODqigiI5LZOz0bfiVKMr7AxohZ/fA4pehRhdorfP+1K2e7FCD6LhEZm62YxGe1uMgKQ42f+X4wFShYk+Zi9rAq8BdwDYRWSgiDzs8snzMGMNLs8M4fTmeSUOaUCblIvwyzHpc9MAX1gIqSt2kSdUyNA8sy1drjma9CI9/PajZDTZNgcR45wSoCgV7Rx9tMsY8BzQHLgLTsjikUPtu/TF+332GF3rWoWnRSJja5X/9CCX8XB2eysNGtg/i5OU4Fu06nXXj1uPg6jnY9bPjA1N5y74FDnt0aM/ktZIi8qiI/A6sA05jJQeVgbDIy0z4zdaP4HcAvuoBJgUeXwyBbVwdnsrjOtf1p6Z/CT5feSTrstrVO0CFhrDuE0jJ4s5CFRxndltPHpb9yyGnt+dOYSfWiKO3jDG1jTEvGmOyrH0kIl4isklEdorIHhF507Z9tYjssL1Oicg82/a6IrJeRBJsw17zndR+hHLFPZkUuBa3mUPArzaMWAYBjVwdnsoH3NyEke2C2Hf6CmvCz2feWMTqnzp/AMKXOCdA5VrJSTB/rPUouvOrDnkLe5JCkDHmWazkkB0JQGdjTCOspNJTRFoaY9oZY0KMMSHAemCOrf1F4Cng/Wy+T56Q2o8QdTmW+YE/U2zF61D/Phi2CEoGuDo8lY/0aVwRf5+ifLHSjnkIdz0AJStp6YvCYv0kOL0D0+t9KFbWIW9hT1JomZPaR8YSa/vR0/ZKux8WkZJAZ2Cerf05Y8xmIDFbV5BHfL/hGOt2h7PU/2P8Ds6Eds9D/2+hSDFXh6bymaIe7jzetjprws+z+2R05o3dPaHFKIhYDae2OydA5RpRB2H5fzB176X/Kn++t2dOSw44tPaRiLiLyA7gHLDEGJO+YMv9wFLb+s/52q7IaH5YuJQ/SrxJxZgwa4RRl9d0lJHKsSEtqlKiqId9pS+aPgpFfKy+BVUwpSTDr+PA05ulQS+w9dglSnp7OuStHFr7yBiTbHtMVBloLiIN0u0eDMyw5zzpichIEdkiIluioqKye3iuuxKfyJffT2OW5z/x94xHhv4KjQa5OiyVz5X08mRIi6r8tut01oXyvEpZiWHPXLhsxxwHlf9smgonNpLS8x3eXXOJWv4l6B1c0SFv5ZTaR8aYy8ByoCeAiJTDGsH0W/bCBWPMFGNMqDEm1M/PtcM7jTH8+vV/eD/+DTxLB+A2YilUa+XSmFTB8VibQNwEvlpzNOvGLUdbHc8bP3d8YMq5Lh6FpW9Cre4sMO04dC6Wp7vWwt3NMXXTclr7aExWB4mIX+oMaBHxBrph65cA+gMLjTH5d9ZNSjJ7pz3Nw+c+4EzZZniPWgplq7s6KlWABJTy5r5GlZi5+QSXrl7PvHGpylan89ZvIe6yM8JTzmCMVS9N3Enu9SEfLwunTnkfejVw3OAVe2Y0n7+59hHwih3nDgCWi0gYsBmrT2Ghbd8gbnp0JCIVRCQSeA54VUQibZ3ReU9CLFemDeSuiGks9bmPSmMXWrfwSuWyke2DiEtM5vsNx7Ju3GocXI+FbTq3tMDY+q01iKD72/waIRyJusqz3Wrh5qC7BLCzTyEDD2bVwBgTZoxpbIwJNsY0MMa8lW5fR2PM4pvanzHGVDbGlDTGlLZ9n/c6oaMjSf6qB8WPLeUDj+E0HvUVbh6O6fBRqk4FHzrV8WPaugjiE7PoyqsYAtXbw4bPISmLOwuV90VHwp+vQfX2JIUM5eO/DlE/oCTd61dw6NvmNCkUzkUATm7FTO3M9fNHGJ74dzo8/A/KFi/i6qhUAfdkhxpcuHqdWVsjs27c+imIOWV1Oqv8yxhY+CyYZLh3InN3nCLiwjWe6erYuwTIJCmISNnbvHwpjElhz1z4phdXk925L+4NmncbSGigYyaPKJVei+plaVSlNFNXHyE5JYvSFzW7gl9dazJbVmUyVN4VNhMO/Qld/kliqWpMXHaIBpVK0q1+eYe/dWZ3CluBLbav6V9bgMJzb2oMrPov/DKMq7530fXK61Sq3Zgn2we5OjJVSIgIT7YP4tiFa/y550xWja2+hbO74MgKp8SnclnMWfj9RajSApqPZM62SE5cjOO5brURJ6zUeNukYIypbowJsn29+VU4PhGTEmDuk7DsXyTeNYD7Y16E4n58+GCIw2/hlEqvx10VqOZbjM9XHs66UF7wg1DcH9brZLZ8adHzkBgH933C9RRh4tJwGlUpTac6/k55e51yeztXz8O0+yBsJqbTP3jm+miOXE5m0pDG2o+gnM7dTRjRLoidkdFsPJpFyWSPotBiJIT/BWf3OidAlTv2zIN9v0Knl8GvNr9sPcHJy3E827WWU+4SQJNCxs7tg6md4fQOGPAtPxQdyG+7zvC37rVppv0IykX6N62Mb/Ei9pW+CH0CPIvp3UJ+cu2idZcQEAKtxpOQlMwny8JpUrU0HWo7b6KuJoWbhf8FX3WHpHgYtojdpTvz9oK9dKjtx6j2NVwdnSrEvDzdebR1IMv2n+Pg2ZjMGxcrC40fhrCf4YodC/Yo11v8EsRdgj6fgrsHP28+wenoeJ7rVsdpdwmgSeFGm6bCjwOgdDUYsYyYcsGMnb6NssWL8OGDjbQfQbncIy2r4e3pbt/dQsvR1pDGTVMcH5i6MwcWWyOO2j0PFRoQn5jMJ8vDaR5YljY1fZ0aSmZDUhuKyAYROSEiU0SkTLp9m5wTnpMkJ8Giv1u3brV6wOOLMSUr8fKcXUReimPSkMb4lijq6iiVokzxIgxsVoX5O05yOjou88Zlg6Bub9jyFSTEZt5WuU58tDUnwb8+tPsbADM2HefslQSe6ea8voRUmd0pfAa8ATQEDgJrRCT1+UnBmcIbHw3TH7T+mmo1Dgb9CEVL8OPG4ywMO81z3bQfQeUtT7StToqBb9ZGZN249VPW7/j2Hxwel8qhP1+D2DPQ5xPwKEJ8YjKTVxymZVBZWtco5/RwMksKPsaYxcaYy8aY94FxwGIRaUm6xXLytUsRVv/B0ZVw70ToMQHc3Nl9Mpq3Fu6lfW0/RnfQfgSVt1QpW4xeDQOYvvE4V+KzWJOqSjOo0hI2fGrdEau85fByq1ZV6/FQqSkAP2w4RlRMAs92re2SkDLtUxCRtCpvxpjlQD/ge6Cag+NyvOMbrBFGMWfgkblWPXogJj6RcdO3UaaYJ/+n/Qgqj3qyfRCxCUlM33g868atx8Pl47B/geMDU/ZLiIUFT4FvTej4MgDXrifx+crDtK1ZjhZBzu1LSJVZUngXqJd+gzEmDOjC/9ZVzp/2L4Jp94JXaRi+1CoihrU+wstzdnH84jUmDW6i/Qgqz2pQqRRta5bj6zVHSUjKolBenbut/oW1E7X0RV6y9C1rUaQ+n4KnNwDfrz/G+djrPNutlsvCymxG83RjzAYAESkhIiVs248bY0Y4K0CHqNDQ6oAb/heUq5m2efomqx/hb93r0Ly69iOovG1k+yDOxSQwf8epzBu6uUOrsXBqGxxf75zgVOaOrbf6MZuPhKotAYhNsO4S2tf2o2k1133+ZPX4aLSIHAeOAcdF5JiIZLnATp5XugoM+MYay22z51Q0by7YS7ta5bQfQeUL7WqVo15ASaasOkJKVoXyGg0B77K6jnNekBhnrbdcugp0+Wfa5mnrIrh0LZFnu7ruLgEyH5L6KnAv0NEY42uMKQt0Au627SswrH6E7VY/wkCta6Tyh9RCeeHnYll+4FzmjYsUg+Yj4MAiOH/IOQGqjK34D1wIh/smQdESgPUZNHX1ETrX9adx1TJZnMCxMrtTeAToa4xJmyVj+/5BYKijA3OW1H6EYxeuMnFQY8ppP4LKR+4JDqBSaW++WGnHZLZmI8C9CKz/1PGBqYyd3GqVNW8yFII6pm3+dm0El68l8oyL7xIg86RgMlpD2RgTB6Q4LiTnSt+P4KrefqVyytPdjSfaVmdTxEW2Hb+UeeMSftBoEOycYRV8VM6VdB3mj4MSFaD7v9I2R8dZdwld65UnuHJp18Vnk1lSOCkiXW7eKCKdgQJRTGXvqSvaj6DyvYHNqlDK25Mp9twttBpn1fXa/KXjA1M3Wv0BnNsLvf/vhjXdv15zlCvxSXniLgHAI5N9TwHzRWQN1uI6AKFAG6CPowNztNiEJMZO30Zpb+1HUPlb8aIePNKyGp+uCOdIVCxBfiVu39ivNtS+2xr50ubptKGQysHO7IbV70PwQKjTM21z9LVEvl5zlJ53VaBBpVKZnMB5MhuSugdoAKwCAm2vVUAD2758yxjDK6n9CIO1H0Hlf4+2DsTT3Y2pq49m3bj1OLh2wXqMpBwvOQnmjwXvMtDznRt2fbnmCDEJSTydR+4SIPPRRzWBpsaYr40xf7O9vgKapquBlC/N3X6SX3ee4rlutWmp/QiqAPDzKUq/JpWZvS2SqJiEzBtXawMVG1sdzikFpnsw71o/yVqbpdf7NwyDv3T1Ol+vOco9DQOoF1DSdfHdJLM+hY+AKxlsv2Lbl291rV+ev/eow5iONbNurFQ+MaJddRKTU5i2LiLzhiJW6YsL4Vr6wtGiDsLy/0C9++Cu+2/YNWX1Ea4lJuepuwTIPCmUN8bsunmjbVugwyJygpJenoztVFP7EVSBEuRXgu71y/P9hmNcTcii+F29PlCuNix9WwvlOUpKsjVJrUgx6y4hnQuxCUxbF8G9wRWpXd7HRQFmLLOkUDqTfVn2TomIl4hsEpGdIrJHRN60bV8tIjtsr1MiMs+2XURkooiEi0iYiDTJzoUopeDJDjWIjktk5uYTmTd094Cub8CFQ7D9O6fEVuhsmgonNlr9CD7lb9g1ZdUR4hOTeapL3rpLgMyTwhYRuaXGkYgM53+jkTKTAHQ2xjQCQoCeItLSGNPOGBNijAkB1vO/4np3A7Vsr5FY6zkopbKhSdUyNA8sy1drjpKYnEV/QZ1eVlntFe/oIjy57eJRWPom1OpujThKJyomgWnrI+gTUoma/pmMFHORzJLCM8BjIrJCRD6wvVYCTwBPZ3ViY0n9TfO0vdIKtIhISaAzMM+2qQ/wne24DUBpEQnI7gUpVdiNbB/EyctxLNqVxXQiEej+NsSe1VnOuckYqyS2mwf0/sj675zO5ysPk5hs8uRdAmQ+JPWsMaY18CYQYXu9aYxpZYw5Y8/JRcRdRHYA54AlxpiN6XbfDyw1xqR2ZlcC0t/zRtq23XzOkSKyRUS2REVF2ROGUoVK57r+1PQvwecrj2CyKpVdpTnUuxfWTYRY/f8pV2ybBkdXWQm31I0fYeeuxPPDhmM80LgS1csVd1GAmcu0SipYi+sYYybZXsuyc3JjTLLtMVFloLmINEi3ezCQ7YHSxpgpxphQY0yon59fdg9XqsBzcxNGtg9i3+krrD5kRzmLLm9YlTtXvuvw2Aq86Ej441VrjZYmj96ye/KKwySlGMZ3zrsjH7NMCrnBGHMZWA70BBCRckBz4Ld0zU4CVdL9XNm2TSmVTX1CKuLvU5Qpq+wofVGuJjQdBlu/gQuHHR5bgWUMLHwWTLK1vO9Nj41OR8cxfdNx+jepTDXfvHmXAA5MCiLiJyKlbd97A92A/bbd/YGFNxXc+xUYahuF1BKINsYUiBpLSjlbUQ93Hm9bnTXh59l9MjrrAzq+BO5Frc5RlTNhM+HQn9DldShb/Zbdk5cfJiXFMC4P3yWAY+8UAoDlIhIGbMbqU1ho2zeIWx8dLQKOAOHAVCD/L+ajlAsNaVGVEkU9+MKeu4US/tDmKdg7HyK3OD64gibmLCx+yRrN1XzkLbtPXo5j5uYTPNisClXKFnNBgPZzWFIwxoQZYxobY4KNMQ2MMW+l29fRGLP4pvbGGDPWGFPDGNPQGKO/mUrdgZJengxpUZVFu05z4uK1rA9oNQ6K+8Ofr+laztlhDMwfY/XL3DcJ3G79WP10eTgGw9hOefsuAZzUp6CUco3H2gTiJvDVGjsK5RUtYT1GOr4ODi7Our2ybJoK4X9ZayT41b5l94mL1/h58wkGNatKpdJ5vyqtJgWlCrCAUt7c16gSMzef4NLV61kf0GQo+NaEJa9r+Qt7nNsPS16zJqk1G55hk0+WhePmJozplD/qiGpSUKqAG9k+iLjEZL7fcCzrxu6eVkfp+QOw40fHB5efJSXA7OFQpAT0+fSW0UYAxy5cZda2SIY0r0pAqbx/lwCaFJQq8OpU8KFzXX+mrYsgPjE56wPq3QuVm1sLzF+/6vgA86tlb8PZXVZCKOGfYZNJy8LxcBPGdMwfdwmgSUGpQmFk+yAuXL3OrK2RWTdOLX8Rcxo2THZ8cPnRkRWwbhKEPnHDSmrpHT1/lTnbInm4ZTX8S3o5N747oElBqUKgRfWyNKpSmqmrj5CcYsfIoqotoc49sOZjuGrHrOjC5NpFmDvaKj3e/V+3bTZx6SGKeLgxKp+t/65JQalCQER4sn0Qxy5c4489dpUus0prJ16DVf91aGz5ijGw8Bm4GgV9p1prJWQg/Fws83ec5NFWgfj55K/lfjUpKFVI9LirAtV8i/HFysNZF8oDa3hlk0dg81dw0Y4JcIXBjunWBL/Or0LFkNs2m7j0EF6e7oxsH+S82HKJJgWlCgl3N2FEuyB2Rkaz8ehF+w7q+LI1Imnp244NLj+4eAR+fwEC21nLmd7GwbMxLAg7xaOtA/Etkb/uEkCTglKFSv+mlfEtXoQvVtpZ+M6ngjXTec8cOGnP2loFVHISzBkJbu7wwOfW19v4+K9DFPN0Z2S7/HeXAJoUlCpUvDzdebR1IMsPRHHgTIx9B7V5CoqVsya0FdbyF6v+C5GbrUVzSlW+bbN9p6/w267TPN62OmWKF3FefLlIk4JShcwjLavh7eluX1ltgKI+VvmLiNVwaIljg8uLjm+EVe9Bo8HQoG+mTT/+6xA+RT0Y3jZ/3iWAJgWlCp0yxYswsFkV5u84yenoOPsOajoMygbBX69Dih0T4AqK+CswZwSUqgJ3v5dp090no1m85wyPt61OqWKeTgow92lSUKoQeqJtdUSsUTJ2SS1/cW4v7Mz2gon51+8vQvQJ6DsFvEpm2vSjvw5R0suDx9veupZCfqJJQalCqErZYjzcshozN5+wv2+hfh+oFArLJlhlogu6PXNh53Ro/3drMl8mth2/xF/7zjKiXRClvPPvXQJoUlCq0Hqqcy1KFPVgwqJ99h0gAt3egphTsOEzxwbnatGRsOBpKwm2fyHTpteTUnh59i4qlPRiWJtA58TnQJoUlCqkyhQvwvjOtVh1MIqVB6PsOyiwDdS+G9b8H1y94NgAXSUlBeaOsoah9p0C7h6ZNp+8IpwDZ2P41/0N8PHK33cJoElBqUJtaOtqVC1bjP8s2mdfTSSwyl9cj4XV7zs0NpdZP8kaaXX3u+Cbed2iA2di+HR5OPc1qkjX+uWdFKBjaVJQqhAr6uHOiz3rsv9MDL9sOWHfQf51IeQha8WxSxEOjc/pTu+0Zm/Xuw8aP5xp0+QUwwuzw/Dx8uT1e+s7KUDH06SgVCHXq2EFmlYrwwdLDnI1wc7V1jq9Am4esOz2VULznevXYPYIKF4O7v04w0Vz0vtm7VF2nrjM6/fWz5flLG5Hk4JShZyI8I976hEVk8AX9k5oK1kRWo2BXb/Aqe2ODdBZlrxmrTh3/2dQrGymTSPOX+X9Pw/QtZ4/9zWq6KQAnUOTglKKJlXLcE9wAFNWHeZMdLx9B7V5Gor5FozyFwf/gM1fWnWeanTKtKkxhpfmhOHp5sbb9zdAsrijyG80KSilAHipZ11SUuD9Pw/Yd4BXKWu45tGVcHipY4NzpNgomD8WyjeELv/MsvlPm0+w4chFXu5VL9+su5wdmhSUUoA1oW1Ym0Bmb4tkz6lo+w4KfRzKBFp3C/mx/IUxVkJIiIF+U8Ej876B09Fx/Pu3fbQK8mVw8ypOCtK5HJYURMRLRDaJyE4R2SMib9q2i4hMEJGDIrJPRJ6ybS8jInNFJMx2XANHxaaUytjYTjUp5e3JhN/22bcQj0cR66/rs7sh7GfHB5jbNn8Jh/6wJuX518u0qTGGV+fuJjElhXf6NSxwj41SOfJOIQHobIxpBIQAPUWkJTAMqALUNcbUA36ytX8F2GGMCQaGAh87MDalVAZKeXvydJdarDt8geUHztl3UP0HoGJjayRSop39EXlB1AH481Wo2RWaj8yy+a87T7F0/zme716Har7FnRCgazgsKRhLrO1HT9vLAKOBt4wxKbZ2qb959YFltm37gUARKRizQZTKRx5qUY3q5Yrz70X7SUpOyfoANzfrL+0rkbDpC8cHmBuSrsPs4VCkOPSZnOXw0wuxCby5YC+NqpTmsTb5u+BdVhzapyAi7iKyAzgHLDHGbARqAANFZIuI/C4itWzNdwJ9bcc1B6oBt6xmISIjbcduiYqyc2q+UspuRTzceOnuuoSfi2XGZjsntFVvD7W6w+oP4JqdS3260vJ/wZkw6PMp+GT9t+ebC/YSE5/Ie/2CcXcrmI+NUjk0KRhjko0xIVgf7s1t/QRFgXhjTCgwFfja1vwdoLQtiYwHtgO39FwZY6YYY0KNMaF+fn6ODF+pQqt7/fI0r16Wj5YcJCY+0b6Dur5hrT+w+gOHxnbHjq6CtROh6WNQ5+4sm/+19yy/7jzF2E41qVPBxwkBupZTRh8ZYy4Dy4GeQCQwx7ZrLhBsa3PFGPOYLYkMBfwAO2fSKKVyk4jw6j31uHD1Op+tsHM95/J32cpfTIHLxx0bYE7FXbKK3fnWhB4Tsmx+JT6RV+ftpk55H8Z0rOmEAF3PkaOP/ESktO17b6AbsB+YB6TODukAHLS1KS0iqYuaDgdWGWOuOCo+pVTmgiuX5v6Qiny15ignL9u5fkKnV0DcrDUX8hpjYOGzEHvWGn5aJOvO4v8s2s+5mHje6x9MEY/CMYLfkVcZACwXkTBgM1afwkKsx0T9RGQX8B+sBABQD9gtIgeAu4GnHRibUsoOf+9ZF4D/Lt5v3wGlKkGLURA2E06HOTCyHNj5k7VwTqd/WKOlsrDu8HlmbDrO8HZBNKpS2vHx5RFi11jkPCo0NNRs2bLF1WEoVaC9t3g/k1ccZv7YNvZ9OMZdhokh1gfvI3MdHJ2dLh6Fz9tCQCN4dAG4uWfaPO56Mj0/XoUAvz/dHu8imbfPb0Rkq61f9xaF435IKZVjozvWwLd4ESYssnNCm3dpawnLw8usl6slJ8GckSDu8MAXWSYEgA+XHODYhWv8p29wgUsIWdGkoJTKlI+XJ890q82moxf5c+9Z+w5qNhxKV7WVv7BjroMjrf4AIjdB7w+hdNalKXacuMxXa44ypEVVWtXwdUKAeYsmBaVUlgY3q0JN/xK88/t+rifZ8SHvURQ6/9OaC7B7luMDvJ0Tm2HluxA8EBr2z7L59aQUXpi1k/IlvXj57rpOCDDv0aSglMqSh7sbr/Sqy9HzV/lx4zH7DmrQz3qGv/Rt15S/SIiBOcOhZCXo9V+7Dvl0eTgHz8Yy4YGCsd5yTmhSUErZpVMdf9rU9OXjpYeIjrNjQltq+Yvo41bhOWf7/SVrvkTfKVaZ7yzsP3OFySvCuT+kIp3rFt4KOx6uDkAplT+ICK/0qkfvSWv4dHk4r/TKvKooAEEdoUYXWPVfaPwQeJfJvYCMgeuxEHPWmnsQexZiz0HsGSsZ7J5tdXhXa5XlqZKSU3hxVhglvTz557135V6M+ZAmBaWU3e6qWIp+TSrz7doIHmlZjSpli2V9ULc34fN2sOb/rDuHrCQnwtUoiDlj+5A/m+7rTdsSr916vJsHlCgPDfpDhxftuq5v1kawMzKaSYMbU7Z4kawPKMA0KSilsuX57nX4Lew07yzez6dDmmR9QIWG0GgQbPgcat8NKYnWB3rMmZs+8G2vaxcyPo93GevDvoQ/VG5m+778/7alfu9dxnp0ZaeI81f5YMkButYrT+/gALuPK6g0KSilsqVCKS9GtA9i4tJDPN7mEk2r2fFIqNM/YPcc+Kbnjdvdi1of5D7loWwQVG1504d96ge+f5arouVESoptvWV3NyY8UPDWW84JTQpKqWx7sn0QMzYdZ8Jve5k9unXWH6alq8DQ+XDl5I0f9l6lslzLwJFS11t+p29Dypf0clkceYkmBaVUthUv6sHfutXmpTm7WLTrDPfY89jFjg5fZzodHce/F+2jdQ1fBjYrmOst54QOSVVK5ciA0CrUreDDO4v3kZB0y9IneZoxhn/M3U1SSgrv9A3Wx0bpaFJQSuWIu5s1RPXExTi+X2/nhLY84tedp1hmW2+5qq8dI6gKEU0KSqkca1/bjw61/Zi49BCXrl53dTh2uRCbwBu/7iGkEKy3nBOaFJRSd+SVXvWITUhi4rJDrg7FLm8s2EtsQhLv9S/46y3nhCYFpdQdqVPBh4HNqvD9+mMcPX/V1eFkasnesyzYeYrxnWtRu3zBX285JzQpKKXu2LPdalPUw413ft/n6lBuKzoukVfn7aJuBR9Gdajh6nDyLE0KSqk75u/jxagONfhjz1k2Hb3o6nAy9M7v+4iKSShU6y3nhP6XUUrliuHtgqhQ0osJv+0lJSVvLfO7Lvw8MzadYES7IIIrl3Z1OHmaJgWlVK7wLuLO8z3qsDMymgVhp1wdTppr15N4ac4uAn2L8Wy32q4OJ8/TpKCUyjV9G1firooleW/xAeIT88aEtg//PMjxi9d4t18wXp6Fa73lnNCkoJTKNW5uwj961ePk5Ti+WRvh6nDYfvwSX689ysMtq9IiqPCtt5wTmhSUUrmqdc1ydKnrz+Tl4VyITXBZHAlJybwwK4wKJb14sWfhXG85JzQpKKVy3cu96nEtMZmP/nLdhLZPlx/m0LlYJjzQsNCut5wTDksKIuIlIptEZKeI7BGRN23bRUQmiMhBEdknIk/ZtpcSkQXp2j/mqNiUUo5V078EQ5pXZfqm44Sfi3Hqe1+ITWDRrtNMXh7OA40r0amuv1PfP79zZOnsBKCzMSZWRDyBNSLyO1APqALUNcakiEjqv9hYYK8x5l4R8QMOiMiPxpj8UVBFKXWDZ7rWYt72k7zz+36+fLSZQ94j+loiu09FszPyMrsiowmLjObk5TgAAkp58c/e9R3yvgWZw5KCMcYAsbYfPW0vA4wGhhhjUmztzqUeAviIVcO2BHARSHJUfEopx/ItUZQxnWry7uL9rDt8ntY1yt3R+WITkth9Mtr68D8Zza7Iy0Rc+N8azdV8i9G4amkebV2NhpVK06hKKYoV0SVjskusz24HnVzEHdgK1AQ+Nca8KCIXgA+BB4Ao4CljzCER8QF+BeoCPsBAY8xvGZxzJDASoGrVqk2PHctfJXuVKkziE5Pp8sFKShfzZMG4trjZWYAu7noye09bf/mnJoHDUbGkflxVKu1Nw0qlaFi5FMGVS9GwUilKFyviwCspWERkqzEmNKN9Dk2jxphkIERESgNzRaQBUBSIN8aEikhf4GugHdAD2AF0BmoAS0RktTHmyk3nnAJMAQgNDc1b0yaVUjfw8nTnhZ51ePqnHczZfpL+TSvf0iYhKZn9p2PS/voPi4zm0LlYkm2zov18itKocinuDa5oJYDKpShXIvfXa1YWp9xbGWMui8hyoCcQCcyx7ZoLfGP7/jHgHdtjp3AROYp117DJGTEqpRzj3uCKfL3mKO//cYDud5XnxMVraX/9h0Ve5sCZGBKTrQRQppgnwZVL061+eRpWKkWjKqV17WQnc1hSsHUWJ9oSgjfQDXgXmAd0Ao4CHYCDtkOOA12A1SJSHqgDHHFUfEop53BzE/5xT30e/GI9IW/+SWpZJB8vD4Irl+KJtkEE2x4DVSrtrUtjupgj7xQCgGm2fgU34GdjzEIRWQP8KCLPYnVED7e1fxv4VkR2AQK8aIw578D4lFJO0rx6WV6+uy7nYhIIrlyKRpVLU7VsMbv7GJTzOLSj2dFCQ0PNli1bXB2GUkrlK5l1NOuMZqWUUmk0KSillEqjSUEppVQaTQpKKaXSaFJQSimVRpOCUkqpNJoUlFJKpdGkoJRSKk2+nrwmIlFATsuklgMK4ozpgnpdqQry9RXka0tVkK8xP11bNWOMX0Y78nVSuBMisuV2M/rys4J6XakK8vUV5GtLVZCvsaBcmz4+UkoplUaTglJKqTSFOSlMcXUADlJQrytVQb6+gnxtqQryNRaIayu0fQpKKaVuVZjvFJRSSt1Ek4JSSqk0+SYpiEgVEVkuIntFZI+IPG3bXlZElojIIdvXMrbtIiITRSRcRMJEpEm6cy0WkcsisjCL93zUdt5DIvJouu0TROSEiMQWsOtaLCI7bXF8bls1ryBd3woROSAiO2wv/4JwbSLik+6adojIeRH56E6uLa9do237QNs594jIu/n02jJsJyLjbOc1IlLuTq/tjhhj8sULa3nPJrbvfbDWdq4PvAe8ZNv+EvCu7ftewO9YS3u2BDamO1cX4F5gYSbvVxZrjeiyQBnb92Vs+1ra4oktYNdV0vZVgNnAoAJ2fSuA0IL4O3lTu61A+4J0jYAv1jrufrZ204Au+enaMmsHNAYCgQigXG79jubklW/uFIwxp40x22zfxwD7gEpAH6xfEGxf77d93wf4zlg2AKVFJMB2/FIgJou37AEsMcZcNMZcApYAPW3HbzDGnC6A13XF1sYDKALc8SiEvHR9uS0vXpuI1Ab8gdV3eHnY4sor1xgEHDLGRNna/QX0y2fXdtt2xpjtxpiIO7me3JJvkkJ6IhKIlVk3AuXTfUCfAcrbvq8EnEh3WKRtm73u9PhsywvXJSJ/AOewfnFnZeO8WcoL1wd8Y3vE8pqI5Nqq8Xnk2gAGATON7c/P3OTiawwH6ohIoIh4YH1QV8nmJdyWk64tX8h3SUFESmA92ngm3V+2ANj+R8iXY2zzynUZY3pg3VYXBTrn1nnzyPU9ZIxpCLSzvR7JjZPmkWtLNQiYkdsndfU12u4aRgMzse6CIoDk3Di3q68tr8lXSUFEPLH+8X40xsyxbT6begtn+3rOtv0kN/4lUdm27XbnbpGuo+6+7B5/J/LadRlj4oH5WLfLdyyvXJ8xJvVrDDAdaF5Qrs3WvhHgYYzZeoeXdXMceeIajTELjDEtjDGtgANYfQD56dryh5x2Rjj7hdW58x3w0U3b/8uNnULv2b6/hxs7hTbddFxHsu7wOorVyVXG9n3Zm9rkRkdznrguoAQQYGvjgfUX2bgCdH0e2DrwAE+sR2OjCsK1pdv/DvBmbv0/l9euEfC3fS0D7ABq56drs6cdeaCj2WVvnIN/wLZYt3Fhtl+IHVijAXyBpcAhrM6n1F8gAT4FDgO7SDfqBOv2MwqIw3ou2OM27/k41rPMcOCxdNvfsx2XYvv6Rn6/LqznppttcewGJmH91Vkg/t2A4lijcsKAPcDHgHtBuLZ0+44AdQvw/3czgL22V26MjHPFtWXYDnjK9nMScAr4Mjf/HbPz0jIXSiml0uSrPgWllFKOpUlBKaVUGk0KSiml0mhSUEoplUaTglJKqTSaFJTKBhFJtk1G2iNWRdm/iUim/x/ZSjMMcVaMSt0JTQpKZU+cMSbEGHMX0A24G3g9i2MCAU0KKl/QeQpKZYOIxBpjSqT7OQhr0l85oBrwPdZEObBmhK8TkQ1APazZudOAiVizjzti1Zj61BjzhdMuQqlMaFJQKhtuTgq2bZeBOliVZVOMMfEiUguYYYwJFZGOwPPGmN629iOxSjb8S0SKAmuBAcaYo068FKUy5OHqAJQqQDyBT0QkBKuCZ+3btOsOBItIf9vPpYBaWHcSSrmUJgWl7oDt8VEyViXN14GzQCOs/rr42x0GjDfG/OGUIJXKBu1oViqHRMQP+Bz4xFjPYUsBp40xKVhrNaSucR2Dtdxjqj+A0bayzYhIbREpjlJ5gN4pKJU93iKyA+tRURJWx/KHtn2TgdkiMhRYDFy1bQ8DkkVkJ/AtVoXWQGCbbfW3KP635KNSLqUdzUoppdLo4yOllFJpNCkopZRKo0lBKaVUGk0KSiml0mhSUEoplUaTglJKqTSaFJRSSqX5f+jXgAvoL3JMAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.plot(X_test, y_test, label='Actual level')\n",
    "plt.plot(X_test, flaml_y_pred, label='FLAML forecast')\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('CO2 Levels')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Forecast Problems with Exogenous Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Data and Preprocess\n",
    "\n",
    "Load dataset on NYC energy consumption. The task is to predict the average hourly demand of enegry used in a day given information on time, temperature, and precipitation. Temperature and precipiation values are both continuous values. To demonstrate FLAML's ability to handle categorical values as well, create a column with categorical values, where 1 denotes daily tempurature is above monthly average and 0 is below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "''' multivariate time series forecasting dataset'''\n",
    "import pandas as pd\n",
    "# pd.set_option(\"display.max_rows\", None, \"display.max_columns\", None)\n",
    "multi_df = pd.read_csv(\n",
    "    \"https://raw.githubusercontent.com/srivatsan88/YouTubeLI/master/dataset/nyc_energy_consumption.csv\"\n",
    ")\n",
    "# preprocessing data\n",
    "multi_df[\"timeStamp\"] = pd.to_datetime(multi_df[\"timeStamp\"])\n",
    "multi_df = multi_df.set_index(\"timeStamp\")\n",
    "multi_df = multi_df.resample(\"D\").mean()\n",
    "multi_df[\"temp\"] = multi_df[\"temp\"].fillna(method=\"ffill\")\n",
    "multi_df[\"precip\"] = multi_df[\"precip\"].fillna(method=\"ffill\")\n",
    "multi_df = multi_df[:-2]  # last two rows are NaN for 'demand' column so remove them\n",
    "multi_df = multi_df.reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "''' Use feature engineering to create a categorical value'''\n",
    "# Using temperature values create categorical values \n",
    "# where 1 denotes daily tempurature is above monthly average and 0 is below.\n",
    "\n",
    "def get_monthly_avg(data):\n",
    "    data[\"month\"] = data[\"timeStamp\"].dt.month\n",
    "    data = data[[\"month\", \"temp\"]].groupby(\"month\")\n",
    "    data = data.agg({\"temp\": \"mean\"})\n",
    "    return data\n",
    "\n",
    "monthly_avg = get_monthly_avg(multi_df).to_dict().get(\"temp\")\n",
    "\n",
    "def above_monthly_avg(date, temp):\n",
    "    month = date.month\n",
    "    if temp > monthly_avg.get(month):\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "\n",
    "multi_df[\"temp_above_monthly_avg\"] = multi_df.apply(\n",
    "    lambda x: above_monthly_avg(x[\"timeStamp\"], x[\"temp\"]), axis=1\n",
    ")\n",
    "\n",
    "del multi_df[\"month\"]  # remove temperature column to reduce redundancy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timeStamp</th>\n",
       "      <th>demand</th>\n",
       "      <th>precip</th>\n",
       "      <th>temp</th>\n",
       "      <th>temp_above_monthly_avg</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2012-01-01</td>\n",
       "      <td>4954.833333</td>\n",
       "      <td>0.002487</td>\n",
       "      <td>46.510000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2012-01-02</td>\n",
       "      <td>5302.954167</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>40.496667</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2012-01-03</td>\n",
       "      <td>6095.512500</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>26.672500</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2012-01-04</td>\n",
       "      <td>6336.266667</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>20.585000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2012-01-05</td>\n",
       "      <td>6130.245833</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>33.577500</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1864</th>\n",
       "      <td>2017-02-07</td>\n",
       "      <td>5861.319833</td>\n",
       "      <td>0.011938</td>\n",
       "      <td>39.020417</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1865</th>\n",
       "      <td>2017-02-08</td>\n",
       "      <td>5667.644708</td>\n",
       "      <td>0.001258</td>\n",
       "      <td>47.305417</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1866</th>\n",
       "      <td>2017-02-09</td>\n",
       "      <td>5947.661958</td>\n",
       "      <td>0.027029</td>\n",
       "      <td>29.242500</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1867</th>\n",
       "      <td>2017-02-10</td>\n",
       "      <td>6195.122500</td>\n",
       "      <td>0.000179</td>\n",
       "      <td>25.048750</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1868</th>\n",
       "      <td>2017-02-11</td>\n",
       "      <td>5461.026000</td>\n",
       "      <td>0.000492</td>\n",
       "      <td>37.175000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1869 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      timeStamp       demand    precip       temp  temp_above_monthly_avg\n",
       "0    2012-01-01  4954.833333  0.002487  46.510000                       1\n",
       "1    2012-01-02  5302.954167  0.000000  40.496667                       1\n",
       "2    2012-01-03  6095.512500  0.000000  26.672500                       0\n",
       "3    2012-01-04  6336.266667  0.000000  20.585000                       0\n",
       "4    2012-01-05  6130.245833  0.000000  33.577500                       1\n",
       "...         ...          ...       ...        ...                     ...\n",
       "1864 2017-02-07  5861.319833  0.011938  39.020417                       1\n",
       "1865 2017-02-08  5667.644708  0.001258  47.305417                       1\n",
       "1866 2017-02-09  5947.661958  0.027029  29.242500                       0\n",
       "1867 2017-02-10  6195.122500  0.000179  25.048750                       0\n",
       "1868 2017-02-11  5461.026000  0.000492  37.175000                       1\n",
       "\n",
       "[1869 rows x 5 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# split data into train and test\n",
    "num_samples = multi_df.shape[0]\n",
    "multi_time_horizon = 180\n",
    "split_idx = num_samples - multi_time_horizon\n",
    "multi_train_df = multi_df[:split_idx]\n",
    "multi_test_df = multi_df[split_idx:]\n",
    "\n",
    "multi_X_test = multi_test_df[\n",
    "    [\"timeStamp\", \"precip\", \"temp\", \"temp_above_monthly_avg\"]\n",
    "]  # test dataframe must contain values for the regressors / multivariate variables\n",
    "multi_y_test = multi_test_df[\"demand\"]\n",
    "\n",
    "multi_train_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run FLAML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[flaml.automl: 11-07 01:56:05] {2600} INFO - task = ts_forecast\n",
      "[flaml.automl: 11-07 01:56:05] {2602} INFO - Data split method: time\n",
      "[flaml.automl: 11-07 01:56:05] {2605} INFO - Evaluation method: holdout\n",
      "[flaml.automl: 11-07 01:56:05] {2727} INFO - Minimizing error metric: mape\n",
      "[flaml.automl: 11-07 01:56:05] {2869} INFO - List of ML learners in AutoML Run: ['lgbm', 'rf', 'xgboost', 'extra_tree', 'xgb_limitdepth', 'prophet', 'arima', 'sarimax']\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 0, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:05] {3297} INFO - Estimated sufficient time budget=93s. Estimated necessary time budget=0s.\n",
      "[flaml.automl: 11-07 01:56:05] {3344} INFO -  at 0.0s,\testimator lgbm's best error=0.0854,\tbest estimator lgbm's best error=0.0854\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 1, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:05] {3344} INFO -  at 0.0s,\testimator lgbm's best error=0.0854,\tbest estimator lgbm's best error=0.0854\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 2, current learner rf\n",
      "[flaml.automl: 11-07 01:56:05] {3344} INFO -  at 0.1s,\testimator rf's best error=0.0472,\tbest estimator rf's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 3, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:05] {3344} INFO -  at 0.1s,\testimator xgboost's best error=0.6546,\tbest estimator rf's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 4, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:05] {3344} INFO -  at 0.1s,\testimator extra_tree's best error=0.0832,\tbest estimator rf's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 5, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:05] {3344} INFO -  at 0.1s,\testimator xgb_limitdepth's best error=0.0472,\tbest estimator xgb_limitdepth's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:05] {3164} INFO - iteration 6, current learner prophet\n",
      "01:56:05 - cmdstanpy - INFO - Chain [1] start processing\n",
      "01:56:06 - cmdstanpy - INFO - Chain [1] done processing\n",
      "[flaml.automl: 11-07 01:56:06] {3344} INFO -  at 0.6s,\testimator prophet's best error=0.0593,\tbest estimator xgb_limitdepth's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:06] {3164} INFO - iteration 7, current learner arima\n",
      "[flaml.automl: 11-07 01:56:06] {3344} INFO -  at 1.1s,\testimator arima's best error=0.6179,\tbest estimator xgb_limitdepth's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:06] {3164} INFO - iteration 8, current learner sarimax\n",
      "[flaml.automl: 11-07 01:56:15] {3344} INFO -  at 10.1s,\testimator sarimax's best error=0.4334,\tbest estimator xgb_limitdepth's best error=0.0472\n",
      "[flaml.automl: 11-07 01:56:15] {3608} INFO - retrain xgb_limitdepth for 0.0s\n",
      "[flaml.automl: 11-07 01:56:15] {3615} INFO - retrained model: XGBRegressor(base_score=0.5, booster='gbtree', callbacks=None,\n",
      "             colsample_bylevel=1.0, colsample_bynode=1, colsample_bytree=1.0,\n",
      "             early_stopping_rounds=None, enable_categorical=False,\n",
      "             eval_metric=None, feature_types=None, gamma=0, gpu_id=-1,\n",
      "             grow_policy='depthwise', importance_type=None,\n",
      "             interaction_constraints='', learning_rate=0.29999999999999993,\n",
      "             max_bin=256, max_cat_threshold=64, max_cat_to_onehot=4,\n",
      "             max_delta_step=0, max_depth=6, max_leaves=0,\n",
      "             min_child_weight=0.9999999999999993, missing=nan,\n",
      "             monotone_constraints='()', n_estimators=10, n_jobs=-1,\n",
      "             num_parallel_tree=1, objective='reg:squarederror',\n",
      "             predictor='auto', ...)\n",
      "[flaml.automl: 11-07 01:56:15] {2900} INFO - fit succeeded\n",
      "[flaml.automl: 11-07 01:56:15] {2901} INFO - Time taken to find the best model: 0.13156795501708984\n"
     ]
    }
   ],
   "source": [
    "from flaml import AutoML\n",
    "automl = AutoML()\n",
    "settings = {\n",
    "    \"time_budget\": 10,  # total running time in seconds\n",
    "    \"metric\": \"mape\",  # primary metric\n",
    "    \"task\": \"ts_forecast\",  # task type\n",
    "    \"log_file_name\": \"energy_forecast_categorical.log\",  # flaml log file\n",
    "    \"eval_method\": \"holdout\",\n",
    "    \"log_type\": \"all\",\n",
    "    \"label\": \"demand\",\n",
    "}\n",
    "'''The main flaml automl API'''\n",
    "try:\n",
    "    import prophet\n",
    "\n",
    "    automl.fit(dataframe=multi_train_df, **settings, period=multi_time_horizon)\n",
    "except ImportError:\n",
    "    print(\"not using prophet due to ImportError\")\n",
    "    automl.fit(\n",
    "        dataframe=multi_train_df,\n",
    "        **settings,\n",
    "        estimator_list=[\"arima\", \"sarimax\"],\n",
    "        period=multi_time_horizon,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction and Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted labels [5378.69   5595.7896 5595.7896 5577.9424 5688.549  5688.549  5422.055\n",
      " 5342.597  5422.055  5464.396  5381.5674 5342.597  5342.597  5342.597\n",
      " 5473.1265 5436.5103 5342.597  5378.3965 5422.055  5592.1016 5872.4897\n",
      " 5667.3687 5257.6274 5314.817  5342.597  5342.597  5643.813  5912.9023\n",
      " 5967.957  5795.3145 5971.852  5912.9023 5884.079  5517.288  5313.4077\n",
      " 5346.9585 5436.3374 5396.2744 5464.396  5857.3247 5429.403  5281.303\n",
      " 4844.5103 5362.985  5493.6    5281.303  5350.9565 5557.2104 4918.1357\n",
      " 4764.0874 5281.303  5411.9106 5281.303  5479.9336 5350.9565 5035.992\n",
      " 4808.9214 5013.9297 5575.4644 5383.422  5308.707  5277.3105 4808.9214\n",
      " 4945.942  5690.7725 5281.303  5310.029  5317.102  5317.102  4846.8096\n",
      " 4764.0874 5192.4863 5380.514  5281.303  5376.619  5969.391  6284.5635\n",
      " 4764.0874 5325.9    5865.0435 5323.8125 5308.707  5356.319  4893.7354\n",
      " 4801.9756 5281.303  5281.303  5281.303  5277.3105 5277.3105 4857.7466\n",
      " 4764.0874 5325.9    5868.8076 7046.5815 7989.6543 7944.1553 4933.812\n",
      " 4763.597  5395.818  5586.2036 5456.4707 4846.8096 5174.2695 5197.3496\n",
      " 4810.755  5293.418  5293.418  5719.2563 6404.9204 6007.378  5108.179\n",
      " 4914.2764 5705.765  5281.303  5357.2964 5529.749  6096.401  6701.786\n",
      " 7702.796  8667.149  8816.328  6901.971  6199.1475 5549.387  5833.8467\n",
      " 6886.0728 7818.458  7301.3193 7237.4644 7281.0986 7598.0854 7259.58\n",
      " 6449.9126 5727.198  6341.534  6131.614  7068.7393 7912.0776 6870.5044\n",
      " 7509.707  7828.836  7472.81   6976.516  6677.66   6611.8164 7022.2773\n",
      " 7132.312  7237.4644 7626.201  8138.9395 8191.993  6542.9155 6912.963\n",
      " 6840.9    7378.3535 8239.682  8600.579  8749.758  8522.787  7852.093\n",
      " 7009.337  6529.1504 6288.1235 7129.577  6607.154  7233.0396 5845.313\n",
      " 5546.1987 7149.515  7869.974  7513.805  7186.382  7480.167  6948.469\n",
      " 5826.4907 6375.343  6155.4995 6759.061  7292.107 ]\n",
      "True labels 1869    5486.409375\n",
      "1870    6015.156208\n",
      "1871    5972.218042\n",
      "1872    5838.364167\n",
      "1873    5961.476375\n",
      "           ...     \n",
      "2044    5702.361542\n",
      "2045    6398.154167\n",
      "2046    6471.626042\n",
      "2047    6811.112167\n",
      "2048    5582.297000\n",
      "Name: demand, Length: 180, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "''' compute predictions of testing dataset '''\n",
    "multi_y_pred = automl.predict(multi_X_test)\n",
    "print(\"Predicted labels\", multi_y_pred)\n",
    "print(\"True labels\", multi_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mape = 0.04057276497360143\n"
     ]
    }
   ],
   "source": [
    "''' compute different metric values on testing dataset'''\n",
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print('mape', '=', sklearn_metric_loss_score('mape', y_true=multi_y_test, y_predict=multi_y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAACH5klEQVR4nO29eXxcV333/z537uyj0W55X+LYceLEdvZANpJACGUP0NDQEFoglKWlD78+T0OhDWVpKWUrgVICAUILYYeEsiVsSSBkcfY9jhM73q11NPvMvff8/jjnzoykmdFIlizJPu/XSy+N7ty5c+5Iup/73YWUEoPBYDAYmmHN9QIMBoPBMP8xYmEwGAyGSTFiYTAYDIZJMWJhMBgMhkkxYmEwGAyGSbHnegGzQU9Pj1y9evVcL8NgMBgWFPfdd9+AlLK33nNHpFisXr2arVu3zvUyDAaDYUEhhNjZ6DnjhjIYDAbDpBixMBgMBsOkGLEwGAwGw6QckTGLepTLZXbv3k2hUJjrpRimSSQSYfny5QSDwbleisFw1HHUiMXu3btpa2tj9erVCCHmejmGKSKlZHBwkN27d7NmzZq5Xo7BcNRx1LihCoUC3d3dRigWKEIIuru7jWVoMMwRR41YAEYoFjjm92cwzB1HlVgYDAbDTLJrKMdvnjww18s4LBixOMz8+Mc/RgjBk08+Oem+n/3sZ8nlctN+r69//eu85z3vqbu9t7eXk08+mXXr1vHSl76UO++8c9rvM9OsXr2agYGBuV6GwTApX77jWf7qv+/H9Y78uUBGLA4zN954I+eccw433njjpPseqlg047LLLuOBBx5g27ZtXH311Vx66aU88cQTs/JeBsORyt6RAiXX42D6yI+lGbE4jGQyGX7/+99z/fXX8+1vf7uy3XVd/u7v/o4TTzyRTZs2ce211/K5z32OvXv3csEFF3DBBRcAkEgkKq/5/ve/z1ve8hYAfvKTn3DmmWdy8skn8+IXv5gDB6ZmFl9wwQVcddVVXHfddQBs376dSy65hFNPPZVzzz23YgW95S1v4Z3vfCdnnXUWxxxzDL/73e/4y7/8S44//vjKWgDe+c53ctppp7Fx40auueaayvbVq1dzzTXXcMopp3DSSSdVjjs4OMjFF1/Mxo0bedvb3oaZ3mhYKOwfzQOwezg/xyuZfY6a1Nla/vknj/H43tEZPeYJS5Nc88qNTfe56aabuOSSS1i/fj3d3d3cd999nHrqqVx33XXs2LGDBx98ENu2GRoaoquri09/+tP89re/paenp+lxzznnHO666y6EEHzlK1/hE5/4BJ/61KemtP5TTjmFL33pSwBcddVV/Nd//Rfr1q3j7rvv5l3vehe/+c1vABgeHuaPf/wjN998M6961av4wx/+wFe+8hVOP/10HnzwQbZs2cLHPvYxurq6cF2Xiy66iIcffphNmzYB0NPTw/33389//ud/8slPfpKvfOUr/PM//zPnnHMO//RP/8RPf/pTrr/++imt3WCYK/anigDsGc5z+uq5Xctsc1SKxVxx44038t73vheAN77xjdx4442ceuqp/OpXv+Kv/uqvsG316+jq6prScXfv3s1ll13Gvn37KJVK06pD8O/mM5kMd955J294wxsqzxWLxcrjV77ylQghOOmkk+jr6+Okk04CYOPGjezYsYMtW7bw3e9+l+uuuw7Hcdi3bx+PP/54RSwuvfRSAE499VR++MMfAnD77bdXHr/85S+ns7Nzyus3GA43JcdjIKP+N3YPz467eD4xq2IhhHgv8HZAAF+WUn5WCNEFfAdYDewA/lRKOSxUXuR/AH8C5IC3SCnv18e5EvigPuxHpZQ3HMq6JrMAZoOhoSF+85vf8MgjjyCEwHVdhBD8+7//e8vHqE0dra03+Ou//mve97738apXvYrf/e53fOhDH5ry+h544AGOP/54PM+jo6ODBx98sO5+4XAYAMuyKo/9nx3H4bnnnuOTn/wk9957L52dnbzlLW8Zs1b/NYFAAMdxprxOg2G+UBun2DNy5LuhZi1mIYQ4ESUUZwCbgVcIIY4FrgZ+LaVcB/xa/wzwMmCd/roK+KI+ThdwDXCmPtY1QogFd+v5/e9/nyuuuIKdO3eyY8cOdu3axZo1a7jjjjt4yUtewpe+9KXKxXNoaAiAtrY20ul05Rh9fX088cQTeJ7Hj370o8r2VCrFsmXLALjhhqnr6G233cZ1113H29/+dpLJJGvWrOF73/seoCyOhx56qOVjjY6OEo/HaW9v58CBA/z85z+f9DXnnXce3/rWtwD4+c9/zvDw8JTPwWA43OxPVcXiaIhZzGaA+3jgbillTkrpALcBlwKvBvwr2g3Aa/TjVwPfkIq7gA4hxBLgpcCtUsohKeUwcCtwySyue1a48cYbee1rXztm2+te9zpuvPFG3va2t7Fy5Uo2bdrE5s2bKxfOq666iksuuaQS4P74xz/OK17xCl74wheyZMmSynE+9KEP8YY3vIFTTz110viGz3e+8x22bNnC+vXr+Zd/+Rd+8IMfcPzxxwPwzW9+k+uvv57NmzezceNGbrrpppbPc/PmzZx88sls2LCByy+/nLPPPnvS11xzzTXcfvvtbNy4kR/+8IesXLmy5fczGOaK/aNKLI5dlGDPUSAWYrYyT4QQxwM3AS8A8igrYitwhZSyQ+8jgGEpZYcQ4n+Bj0spf6+f+zXw98CLgIiU8qN6+z8CeSnlJ8e931Uoi4SVK1eeunPn2BkeTzzxROViaFi4mN+jYb7wlTue5aM/fYI3nr6CHz6whyc/fAmWtbC7DAgh7pNSnlbvuVmzLKSUTwD/BtwC/AJ4EHDH7SOBGVErKeV1UsrTpJSn9fbWnQpoMBgMM8b+VIFoMMDxS5Iq2J0tTv6iBcys1llIKa+XUp4qpTwPGAaeBg5o9xL6+0G9+x5gRc3Ll+ttjbYbDAbDnLFvtMDi9gjLO6PAkR+3mFWxEEIs0t9XouIV3wJuBq7Uu1yJclWht79ZKM4CUlLKfcAvgYuFEJ06sH2x3mYwGOYBUsojppByKudyIFVgcTLC8s4YwBEft5jtCu4fCCEeB34CvFtKOQJ8HHiJEGIb8GL9M8DPgGeBZ4AvA+8CkFIOAR8B7tVfH9bbDAbDPODmh/Zyxr/8mrLrzfVSDpmfPLyv5XPZl1KWxbKjxLKY1ToLKeW5dbYNAhfV2S6Bdzc4zleBr874Ag0GwyHz1P40/ekiuaJLe2xhdxC6f+ewOpeSS3u08bl4nuRgWolFImzTEQuyZ+TILsxb2L9Zg8Ew54zkywDky+4ke85//ErsySyLwWyJsitZnIwA0JsIM5gpzfr65hIjFoeRQCDAli1bKl87duzgd7/7Ha94xSsavmbLli288Y1vHLPtLW95C7FYbEzB3t/+7d8ihKi09q5tOlgPv025v5Y3v/nNh3BmM8eOHTsqdSaGhYGXPsArrD8eIWKhXEmO2zxucUDXWCxuV2IRC9tkSwv//JthxOIwEo1GefDBBytfq1evbrr/E088geu63HHHHWSz2THPHXvssZViOc/z+M1vflOp4m6Vyy67rLKWb3zjGy29RkqJ582eb9qIxcLjRQf/h8+HrqWYz8z1Ug4JKSW7hlqzLPxWH4vaVPuaeChArnhkt68xYjGPufHGG7niiiu4+OKLJ1RRv/GNb+Q73/kOAL/73e84++yzK40ID4VPf/rTnHjiiZx44ol89rOfBdQF/LjjjuPNb34zJ554Irt27eLf//3fOf3009m0adOYNuTf+MY3KpXoV1xxBdC4hfptt91WsWxOPvlk0uk0V199NXfccQdbtmzhM5/5zCGfj2H22ZC/H4BiYWEHeEdy5Yp1MKkbSrucuuNaLI4Cy+Lo7Dr786th/yMze8zFJ8HLPt50l3w+z5YtWwBYs2bNmP5O9fjOd77DrbfeypNPPsm1117L5ZdfXnlu/fr13HzzzQwPD3PjjTfy53/+5y31YRp//N///vcAvPe972XTpk187Wtf4+6770ZKyZlnnsn5559PZ2cn27Zt44YbbuCss87illtuYdu2bdxzzz1IKXnVq17F7bffTnd3Nx/96Ee588476enpqfS4atRC/ZOf/CRf+MIXOPvss8lkMkQiET7+8Y/zyU9+kv/93/+d0rkY5ojMQVa7qltCbXfihciums6xziST74aySix6cttg50PEQyeRKx3ZlsXRKRZzhO+GaoWtW7fS09PDypUrWbZsGX/5l39ZmXPhc+mll/Ltb3+bu+++uzKLYipcdtllfP7zn6/8/B//8R+89rWvJR6PV45/xx138KpXvYpVq1Zx1llnAXDLLbdwyy23cPLJJwOqrfm2bdt46KGHeMMb3lDpT+WvtVEL9bPPPpv3ve99vOlNb+LSSy9l+fLlUz4Hwxzz3O2Vh6XywhaL2tTXktPcshjKlgjZFtE7PwlP/i/JE28lWzSWxZHHJBbAfODGG2/kySefrMQ1RkdH+cEPfsDb3/72yj6XXXYZp556KldeeSWWNbseRV9AQPl23//+9/OOd7xjzD7XXntt3dc2aqF+9dVX8/KXv5yf/exnnH322fzyl6bWcqHhPXtbxZddKi7sbCA/XgGTWxaD2RI9MRux4w6QHivdXWSL0dle4pxiYhbzEM/z+O53v8sjjzzCjh072LFjBzfddNOEud2rVq3iYx/7GO9617tm5H3PPfdcfvzjH5PL5chms/zoRz/i3HMnlMrw0pe+lK9+9atkMiqguWfPHg4ePMiFF17I9773PQYHB4Fqq/VGLdS3b9/OSSedxN///d9z+umn8+STT05oy26Y38hnb8OR6jJSLB05loUzacyiyOnR3ZBX7fSXl3eSL7u4k4jMQsaIxTzg17/+NcuXL6983XHHHSxbtoylS5dW9jnvvPN4/PHH2bdv35jXvuMd72Dt2rUTjpnL5cYc89Of/vSk6zjllFN4y1vewhlnnMGZZ57J2972toqrqZaLL76Yyy+/nBe84AWcdNJJvP71ryedTrNx40Y+8IEPcP7557N582be9773AY1bqH/2s5+tzB0PBoO87GUvY9OmTQQCATZv3mwC3POd1G4CqZ08KI8FoLzAxaI2ZlGaRCyGsiVeIB5TP4gAi4vPAkdGrUkjZq1F+Vxy2mmnya1bt47ZZlpbHxmY3+M8Yu+DcN35fN89j9cHbueHZ3ybS//kZXO9qmnz4k/fRipfpj9d5Bt/eQbnrW/cvfqcf/sNX7b+leOjKQgE2eN1cvaud3LPP1zEIl2otxCZkxblBoPhCKes7sTTUvnqy6WFG7OQUrJ7OMeaHhWbmyx1Np3NsTb/MKw5DxYdT2fmGYAjOn3WiIXBYJgeWixGUV1Xy+WFKxYDmRKFsscxFbFo7HHJl1yOKT9DyCtUxCKW30eCHNkjuDDvqMqGklKihvMZFiJHost0QVNWAeG0VGLhLGCx8Nt3rOjS59KkS8FgtsgK0a9+6D0OLHUZXS92H9FicdRYFpFIhMHBQXPBWaBIKRkcHCQSWbj+4CMOLRa+ZeEs4DoLv8jOb9/RzA01lC2xWKiMP9qWwCIVQ1tv7SZ3BLuhjhrLYvny5ezevZv+/v65XophmkQiEVO4N5/QbqiC1QaA6yxcy8IXiz4dnG7mhhrMllgshnGDCQKRJIQSeHaU45xdZI/gKu6jRiyCwWClcthgMMwA2rKQ4TZwwF3AbqjBCWLRxLLIlFgshnATSwgAWBZucgWLi0Okj+Aq7qPGDWUwGGYYbVmISBIAzynP5WoOieFsiYAl6IqHgOYtyoeyJZaIIaz2ah2UiLTTRu6ItiyMWBgMhulRyuFhEYgsfDfUYLZEZyxIOKguic0si4FskcViiEBHdSSAFU3SJvJHdMzCiIXBYJge5TwFwsSiqs5CLmCxGM6W6IqHCFq+WDS2LEbSOXrFCKKtallYkXaSIkfGZEMZDAbDOMo5CoSI6Qw1z124YjGULdEZCxEMqNT6Zr2hnNGD2HiQrIoFkSRJkT+iByAZsTAYDNOjnCcrwyRiWiychXuhHMwW6U6ECFhKLMpNGgJaGd2fLVkzmTKcJEHeVHAbDAbDeNxSlpwMEY9pN5S7gAPcuTJd8RBCCIIB0TRmEc7tVw+SS6obI0nClCgWF/a0wGYYsTAYDNPCKeTIEyIRV0V5LFCxcD3JcK5EV0xlQgUDVkM3lJSSWEGNBR5rWbSr5/Ojs7rWucSIhcFgmBZeKUuBMPFoxN8wtwuaJiO5ElJSSZu1LdEwwD2ad+jyBnFFEGLd1SfCKiNMFI1YGAwGw1jKOfIyRDikWmRYnjNpt9b5yHBOiVxnvGpZNDqP3SM5FotBitE+qO0zp2tNKB65g7uMWBgMhulRzpMnjG2rRhC2cCkswOE/gxklFt1xJXrKDVXfstg7UmCxGMarSZsFIKzEIlA2loXBYDCMwXLy5AkRtAO4wiaIuyAnxfmWRcUN1STAvWc4x2KGsGsK8oCKZWGXMrO30DnGiIXBYJgWwsmTl2GCAQtpBbFxKZYXnhvK7wvli0UoYDVMnd0zkqdXjBDuXDL2CW1ZBB0jFgaDwTAGyymQpyoWQZwFaVkMZfyYRRBchxXsp+zUF729wxnioojQ4lAhorKhwm4Gr0mNxkLGiIXBYJg6UhJwlRsqFLCQlq3EYgEWpQ3lSrSFbcIBC378Tr6ceQ84hbr7Dg6NqAeh+NgndDZUG7kFKZitYMTCYDBMHbeEkB4FGSZoi4obaiEGuIeyJZUJ9eC34JHvEqKM5dQvrhtOjagHodjYJwJBnECENpE/YqflGbEwGAxTp5QFUAHugAWBIEGxMAPcQ9kSq6IF+NnfQUDFLWSdPleFskshp1NjQ4kJz5ftNt2mfOF9Bq1gxMJgMEwdPfgoT1h1ag2omMV0LYutO4Z497funxN//1C2xHGhATWf49iXqI11qtH3jOSJoUfHBmMTnvdCCdpEzlgWBoPBUMEXCxkiaAtEQLmhpmtZ3LFtgJ8+vI/0HFxoR3Jl+mxlKVX6PdWxLPYM54mhYxnjYxaAF0qSJHfEzrQwYmEwGKaOnpLnZ0OJQJAgLoVpps76cyDmIuaRKTp0WTrlNbFYfa8nFiN54sIXi4luKBlOkhD5I3Za3qyKhRDi/wghHhNCPCqEuFEIERFCrBFC3C2EeEYI8R0hREjvG9Y/P6OfX11znPfr7U8JIV46m2s2GAwtUOuG0mJhH0I2VLqg3D6HO5tKSkm26NCJjkW0KbEQddxQe0fyxIUWkfEBbtR42TZMgHvKCCGWAX8DnCalPBEIAG8E/g34jJTyWGAYeKt+yVuBYb39M3o/hBAn6NdtBC4B/lMIEZitdRsMhhbwLQupUmctO3RIbijfsjjcLpyi4+F4kqRMg7Ag3qOeqNMUcc9IniVRLQR13FAi0k6bMG6o6WIDUSGEDcSAfcCFwPf18zcAr9GPX61/Rj9/kRBC6O3fllIWpZTPAc8AZ8zyug2GuWNwO+zeOteraM4Yy0LFLELCoThNsUgX1EX4cGdT+e/bJkch2gm26g9Vz7LIFh06bS0WwYliEYgmaSO3INOHW2HWxEJKuQf4JPA8SiRSwH3AiJTSt9N2A36TlWXALv1aR+/fXbu9zmsqCCGuEkJsFUJs7e/vn/kTMhgOF7/+MHz1Etjxh7leSWO0ZVEQarqcEgtv2hd7/6J9uC+0vsso7qRUy/GA30F3oliUHI+4pbOh6lgWgVgHcVEkXyjO3oLnkNl0Q3WirII1wFIgjnIjzQpSyuuklKdJKU/r7e2drbcxGGaf4ih4ZQr/82c89fhDc72a+mixcEQUIQRYQUKHUGfhu6EOd8zCf9+Yk4JoV6XOol7qbMn1iIvGqbN2TLX8cI/QAUiz6YZ6MfCclLJfSlkGfgicDXRotxTAcmCPfrwHWAGgn28HBmu313mNwXDkUcohu9cTLI8y8PuvzvVq6qPdUOWAHnwUCCmxKE0zG2qO3FC+WITLI9qyCAJgyYkxi2LZI05BCYU18dJp6c6z8ggdgDSbYvE8cJYQIqZjDxcBjwO/BV6v97kSuEk/vln/jH7+N1JKqbe/UWdLrQHWAffM4roNhrmlnKXYvpoSQaz5On1OWxZuRSxsgsKl4BymbKh7vwLXXzyt96rFF6lQaQRinRXLwmpoWRTquqCASudZCkYspoSU8m5UoPp+4BH9XtcBfw+8TwjxDComcb1+yfVAt97+PuBqfZzHgO+ihOYXwLullEdmBMlgACjnKYgILhZ48/RPXVsWni8WVpAQDoVJLva7h3MTtrmerLTIaNmy2P8I7H2w5eU2QlkWErswrC0LJRZC1o9ZRCnWdUEB1Wl5hyoWmX6Q869z7axmQ0kpr5FSbpBSniilvEJnND0rpTxDSnmslPINUsqi3regfz5WP/9szXE+JqVcK6U8Tkr589lcs8Ew55TUuFIXC+Q8nQ9RzlESEUK2zmJvoYL7kd0pzvm33/LY3tSY7bVFbC2LRSkLbvGQxTRTdIhRRHglHbNQbqhAgwB3VBbqFuQBFcvCKh3CaNX0fvj0BnjmV9M/xixhKrgNhvlGOUdGhnGxEN48LfAq5ylZKm0WAMtWw48azIEA6M+o6ufnB8daF34mFEzBDaUbGfoWDqDu6Hfe2drrNZmiQ5fQF/cay8LGwR3Xp6roeEQo1C3IAyozLQKl6VsW+cHd4Dk8/uBd0z7GbGHEwmCYb5RzpF1tWXjz1LIo5SgKVb0NQCCkxaLxxd5K7eIXob8nO7hrzPZMYTqWhW7PUTt3Yuv18PWXQyFV/zV1yBRqxaKaDRXEmTBateh4RGSTmIXeLpyJrrZWOTAwAMDdDz3Cv//yyWkfZzYwYmEwzCdcB9wSadfGJQDzNTyn3VB2RSxUu49mY1Ujw0+xwdqFffCxMdszxarLZ+qWRc2FeehZ5bZLtZ4smSk6LA5p66TGDRXCwRlnWZQcl4iXr1uQp16ktgfK0xeLUnYEgPXRNF/47Xa298+fMa12oyeEENcCDaMsUsq/mZUVGQxHM/pCM1wOKjeUnL9uqIIIE6pxQwWk0zQbSpZ0BlVmYMz20WlZFvqCXK6xLEa0xTK6F/pOaOkwmaLDEjsLDmPcUCEcNVo1XN236HiEmlkWOvBtu9MXi3JOWUUnJDKQht891c/a3gYxksNMM8tiK6riOgKcAmzTX1uA0KyvzGA4GqmIhY0n53M2VI4CtW6oIIFJLAv/3GRucMxm3w0lxBQD3DXHBCDli8UULIuCwyJbHyNWtSyCOJRrXIBSSkquR8jLNxYLK0BJhAm49afstYJf0NdWPMCxixL87qmDAE3de4eLhmIhpbxBSnkDsAl4kZTyWinltah6iS2HaX0Gw9GFvvgNFG0cLMS8zYbKU6wVCyuILZv3hhLaGhCF4THb/cK47nho0tTbCuNjFlJCard6PLq3tWOgMrF6rAwgINIBVgBPBAgKh7Jbdaw4nkRKCLpNxAIoBaKEZkAsArmDXLSuk7ufHeLRPSlO+8iv+NEDu6d93JmglZhFJ5Cs+TmhtxkMhplGX1D7ixYeFmLexizyqomgXQ1wA5SdiSmn1deocwsVx4qFX5DXkwi33rF1vGWROVgRDjkFyyJd0LMsIu0QUF55T6ipf05NgLvkeAgmsSyAshUj5E3fDSWLKtgukLxkpSoEvOL6u0kXHXYPTV+EZoKGMYsaPg48IIT4LSCA84APzeaiDIajFp0K2l8I4Abms1jkKNBbjVnoC63rlJFSqn5R47AcdW6R8thspUzBQQglFplWZkG4jqqxgGrqbKqaYeWM7CHY4mlUZlnEuivbZEAVGNZaFkXHI4qupm9UlAc4doxwvtDwM5iUYrVGY3MySzwUYDinxHS61fEzxaRiIaX8mhDi58CZetPfSyn3z+6yDIajlLK6Y854YdxAYF4X5eVlaIwbClRguOR6hO2JI2csnVIad0cpu17ltemiQyJkEw8HGMi00LFVf0bqsRaLkecBeM7rY8VUYhZFh6SdVvEKjWcFJ6TOlhyvOn+7iWXh2lFiFFSabXDqY3esUjX7KZjdx5+/4HhG82V+8tC+affdmilaTZ0NAP2oYUXrhRDnzd6SDIajmMqcCFVnYc3jbKgc4TGps6CK2RoV5vmWRafIMJyt9rxKFxwSEZtoMNBagLtURyy0ZXGfPA4rva/l08gUHNo83XFWIy01ItapsSxKjkesyUhVH8+OEReFaXfPDZQz7Be6a/boXt7/suP510s3EQla89+yEEL8G3AZ8Bjg/xVI4PZZXJfBcHSiL4R5dAX3PLYssiJUreCuiIVLoeySjEx0BFXFIs1ApsSipOorlSk4dIVcVrs7uLPUNvl714qFH+AeeZ6clWBbeRlW6Xblzgk3P5aUkkzJIRFOVSfkocVCKAup8pauW2NZNHZDecE4MfaSK7vTCuzaToaBQC+LRXZMoD4SDMz5UKVWYhavAY7zezgZDIZZpDKuNIwVCDAve2a6ZfAccpYaqQpU3FBB3Ibps7bOEuokw7ZMAT9vJlN0eL38JVc8cwM3el+Z/P1rXDWVAPfILgaDi9lX0BbC6D7obS4WuZKLkB7x8mBl9jaADIRUUV6NWBTKHjF8y6KxG0qG4sQoTtuyCLlZUoEOSCwdkwI8H8SiFTfUs9ByvMhgMBwK2q2SI0wgEJyfloW+QOdqYxa+ZSEa94fyxSIsygynRirb04Uyq+VebFmmyzkw+fuPcUPpC3hqF/3WIvZLXywmj1tkig7djGJJF9qWVJ8I6GyomgruSntyaFzBDRBKEDsEN1TEzVK2E5BcOs6ysCg0q2E5DLRiWeSAB4UQvwYq1oWp4DYYZgF9ISxbEaxAAMudh5aFFrSsVxvgVpeSIE7DO2DbrVZb54YPAusBFeDuk6r4rE8OjAl+16VGLJxiFltKGNnFfnsd+/DFYvJai0zRYZEYUT/UWhZWiCDj3FAtBritUJw4xWkPcYrIPE4wAckEPHdHZXt0HlgWrYjFzfrLYDDMNuU8HoJ4LI4UAQTzcPiRtiwyXpBOe2zMItik82zQq9YJ5EerLT8yBYceS4nFEjFIvuy2LBalQhY7PwylNHvsXg5KHSloRSwKDouErvlIVMWCQIggRQrjUmdbcUOJcJyYKJKb5hzumMzhBbVlkd6nKvitAJFgoLW04lmkldTZGw7HQgwGA5UGfZ3xMLIUwJqPbqiSLxYh+gJji/JUNlT9O+CgV2BQdNEthyinq2KRLpTptFU2/hIxSKFUP0Beff+qWJQLWTUDAtjrdlIkRDHURXh08mrnTNGhzxeLGstCBIKERJb0eMtCTG5ZBCIJva5pNAB0HaIUkeE2SC5TTSTT+6F9GZ0ix1BpGnUbM8ikMQshxDohxPeFEI8LIZ71vw7H4gyGo46yav0dCQWQIqD86fMNP64iQ9hWnQB3A8si5BUYCKi00HJmgLfdcC//9osniZeHCeqZ18vE4ORV3FosUjKGU8xVWpL3O1EAspG+1t1QjKgfEn3VJ+xwgzqLyS0LO6KC6uX81MWiqJsIinAbtK9QG1O7oZTl33a/ifPzczsQqZUA99eAL6L6Ml4AfAP4n9lclMFw1FLKURARwrYFwkIwH8XCD3CHCdpjK7ht3Ib9oUJegWFbicWB/Xv51RMH+eLvtrNc9Ff2WcLg5P5+nQ01INtxS1WxGHBUKm4u2AXZ/oYv98kUlGXhRrrArumN6ge4a+ssalNnm1Rw+2LhFKY+LS83qqycQKQNOnyx2AVDzxH1svQ5rRcbzgatiEVUSvlrQEgpd0opPwS8fHaXZTAcpehurpFgAM+y56cbqlI4GJ6YOisaF+WFZYERLRadZHjV5qUc19fGMqFcUtn2dSwVrYhFljIB0sRU2/Oiar7XX1b9xLOBdsgNsu1Aumm3VhXgHkbWuKAAhB0miDvGsiiWVVGeZ0fBalyZHYz5YpFtuE8jcukRAKxoEtqXq42p3TC8A1CV73NJK2JRFEJYwDYhxHuEEK9FNRM0GAwzTTlHnnDFsrDmsWWRJzRmUh40qbPwPMKUKAYSFO02jokX+ehrT+Qzl23h+Ki6o870nc4SMUSh2KQZISBLWXIyTF6GkaVCxbIY8dQdfzrQjswN8vLP/Z7v3Lur4XH8bChRmzaLilmoFuXjUmcpNLUqAEJRJRbeNGIWRT34KBhtVwWFkQ5lWWixaJOHMNt7BmhFLN4LxIC/AU4FrgCunM1FGQxHLaUceUKEbUvHLOavZTF2noXvhmowAEkLjGtHCbf18Or1EZKRICcsTfKeU8IQ6cDpPo6wKONmmruQnEKGHBHyhBBOHgojAKTRYmElEaUswi2wc7BxB1g/wG0lx1oWlj2xKK/keERFsWm8AsDWAW6vNHWxKGSV6IXiHWpD+4oxlkVSppGy4Ty6WaeVbKh79cMM8BezuxyD4SinnCMnlRsKYWMxH8XCrzKvafdhVdt91LUsasQCuxvyQ9XnRp6HjpVYHcr1IlN7gI0N394ppMnJMAVCCGcUCqN4gQglXTucEqoyvJM0B9ONU1jzhSK99SwLO6znWdS4oRyPJLlJW4hUxKQ4dbHwp+RFEu1qQ/ty9dnoAVgdZKbdoHAmaCUb6jQhxI+EEPcLIR72vw7H4gyGo45yjqxUbihpWQTmtRsqTMgeW8EdapQ6q1/jBWKqw2uuVix2abFQQd1Aunnaq1vIkNWWRcDNQyGFF66O3BnRYtEl0hwcLTQ6DIH8EAHkmLRZAGGHdDbU2DqLDpFB1HSnrYsvFqXmMYvXf/FObrzn+THbnJyKSUTiWiw6tGUxslP9KDLNJxHOMq0U5X0T+L/AIzAfb3MMhiOIcp6cF1Itvue5GypPbeqsruBu1O5D12Z4wajq8Nr/pNoupbp7XnsBwU4lFsFs866xsqjcUEUZwvZUzMIJVu/4h6V63CnS7G1iWUSKqhBwvFj4bqjxqbNdIoOIHd90bX5HWlFuLBaeJ7nv+WE2Lk2O3V5QYhFP6sLC9uVQTFWyvzpJk3Jc2pt1X5JSzaedBVqJWfRLKW+WUj6ns6F2Sil3zspqDIajnVKWtBciErTACsxbN5QXCCOxarrOqgB3NCDrt6XwLYtgDDpWQmoPHHwS8sNqPkX7CiLtvRRkkHC2eY2ELOXIygglK0zQK0JxlJJdFYtBTz3uorllEcn7YjHWDVWNWYxtUd5BFqKT9JLVloVVbhIrKTlIyZh2IlCdkhdv891QOn1WuuSiS4iIMoXcJO6tL54NP3h7832mSSuWxTVCiK8A43tD/XBWVmQwHMXIcp6srFoW89MNlVcppFAzVlXd7UYDXn3Lwr94BqNw5jvgnuvgp/9ftZ5g6clEgjbPyR6iueaz1UQ5S54+gpE4oVIRCilKtrqjjwYDDEr1uFOkyZZcMkWHRHjipS5eHlQPxlkWqt2HQ7mmL1ex7NAuMmPmXtTFjuAhCDiNxWI0r7K9JnxOxVEyMkIipGs+fLEAUl0nEduzj3JmEOihIblBCEaar3GatCIWfwFsQHWerZ1nYcTCYJhJPA/h5MnLMNGghdSWxbRHdM4W5RxuQInF+DqLiOWRruNXl6UsAsCOq9kRL/4Q/O/fwk7ggg/CqhdgAYOig8WlwaZvb5WzZGWEcDSBXXIhN0ghugGArniIfkdd1rqEulM/OFog0Tsx2z9R0llX8UVjnwiEsITEcaq9mEQpg407uWUhBEURbSoW6YLDieJZQoXo2PMqZciJWLUuoaMqFtmeTbDnFpxMk89GSpU4MNkap0krYnG6lPK4WXl3g8FQpaZ+ocO2ECJAAA9PQmAeaQXlPF5A3b2OT52NBty6qbNuIasuNv7goFOuhGd/B52r4Ly/q+yXFzFsp7mrxXZy5AizKJ6AFJA+QD5+GgA9iRA5R1Kwk3Q6WizSRY6pIxbri49ywF5KX231NlSsJNxqE8dgaUQ9mCzADRStKLbb3LL4Zuhf+MPQq4ELK9utcpa8qKnjiC9SIixdij0nqSU1E4tSVq15MutnmrQSs7hTCHHCrLy7wWCoUpllESESDCAtlTrrePMsblHO41TEYmzMImx5dTN2nKIK+ApfLCwL/vQGeMmHxwRkS1aUoNskk0hKgm6evIgSi2oBcPJkLRUr6IqHKJRdsnZHxbI4UC9uMbqPTeWH2Np24cTn9Ll4TjU4HiqptNZW7tpLVpSQm2/4fCabo13kCI8TxaCToWjVWBuWBe3LILkc0aZ6V8nalOPx+M+1IGjToRXL4izUPIvnUDELAUgp5aZZWZHBcLSiM2jyOnUWK4CNh+vNXSFWXUrZGrGY6IaqlzrrjheLRocONL/Q4hSxcHHtGMFItUAuK9TjzniIJ/enydrtdAl1Me6vlxH1yPewkDzY+dKJvYu0WEinWkkeKo+oBy3ctZcDUUJN3FD5jKpYt7yxlepBJ0sxMK7ob8kWkB52XL2vaCoWwy2vcTq0IhaXzMo7GwyGsdSkpIbtAMJSbqjifBOLch5HxyyqYqEKxcINUmc9LRZWuHmnoJIVI+Q0EQs/q8qOEYpWhSdNjGgwQDxkUyi7jAaTLArsIWxb9QvzHv4uj4p1pOOrJz6n3VCyxrIIl7Vl0cJdu2PHCBcaZ2EV0+qCH/DGrivsZslFxh3/0i8DEBpQVpLwBaEefu3KLMUsJnVD6TTZFcCF+nGuldcZDIYpomsRcoTHpM667jwUC2ucG0oIsIKELa9u6mxFLCbJ1HHtGGGZV8HaeuiaAxmKEY5WhWdUxoiFAkRDAfJllxHRRgdpFiXDE91QBx6DA49wszy3fjW0b1m41Tv/aLl1N5QbiBH2xr1n/9NwwyuhmMHJjai38cYOtgrJAp497vOxQ2CHCEdj5GQYq9BELGbZDdVKBfc1wN8D79ebgpgW5QbDzKPvmguElWUhAgSFO2YW9LygnKNkjXNDAQSCjS2LkmpjEgpO4swIJbCQ1VTb8fiV0aEE0VhVLFJejFhYTZQrlD2GZBvtcpS+RJiDo0VGC+VqzcVD3wbL5qbyWU3FAqd6MY/4HV9bEYtgjBiFMUV97L4HnrsdBrfhVsRirBvKlg6eNS7Y7r+/HWCYBIHiSOM3rlgWcxfgfi3wKiALIKXcC0zSIMVgMEyZmjkR4aBVqYr2vHlWa1HOU9ZiUWn3ARAIErLqi4UsZcn53XSbIPQ8iIbtMvR2EU4Qi1fFYsiLEg/ZyiIDDjoJQpRZ0eZxIF3grV+/lyu/dq/qs/TI95FrL+KAm6jsPwY/G6rmYh5zRlWmUqBJ9bR/DuEEMVFgMFNjOfjnkxtC6i65ATnWsrBxkIEGYhEMkJIJgs3EIq+fmys3FFCSqtWhBBBCNG+7qBFCHCeEeLDma1QI8bdCiC4hxK1CiG36e6feXwghPieEeEb3nzql5lhX6v23CSFMx1vDkYm+oOQJE9ExC2BMvv+8oJyjbKnZEWMsCytICLdhBXeB0FhxqYPftZVig3bc2g1lh+PE4jUtPtyIckNpS2FPScUzVkUKPNuf5d4dw2zvz+A9ewek91Lc+KcAlf3H4F+wa1Jn494o2UBy4r51iCfaiVNg28Gac/C70OaHK11yx1sWQek0FKOwbTEsE9UU3nrkh1S7kfGpwDNEK2LxXSHEl4AOIcTbgV8BX57sRVLKp6SUW6SUW1CtzXPAj4CrgV9LKdehqsKv1i95GbBOf12Fms6HEKILuAY4EzgDVVE+O9JpMMwltQFuHbMA8Jzm8x0OO+U8JeGLRU0BSCBIsIFl4XfTDQWad0wNRicZS6rjOsFIgkSialkMOlHiYbty8d9VVJbP8nDVnVVyPIoPfAvCSbKrXgzQwA2lLtheuSoWCS9N3m5NLDraO4hR4Kn9tWJRtSwsPawpyFjLIki5oWVhWYJR0VYNtNcjNzRrLihoLcD9SeD7wA+A44B/klJeO8X3uQjYrgPkrwZu0NtvAF6jH78a+IZU3IUSpyXAS4FbpZRDUsph4FZMhpbhSMRRPvWCTp21AvPQsvBccIsURSPLwqs/VrWkCukmsyxCMXVBzuipcRPeXotFKJogFKmKxYATJhoMVC7+g546zvKwEuDLz1wJSILbfg4bXkEedVGu74ZSzxWL1SB1QqYp2O1N1+4TjScJCZdn99UEo32xyA9hl7RYyOpNgJSSIA7CauzmSlttRJwmYpEfhtjs3UdPmjorhOgARoDvAk9LKZustiFvBG7Uj/uklH5byf2APyl9GVA71mq33tZou8FwZKHFokiQSDBAxo9ZuPNILCpB+HGpswABu+FYVcvR42InEYtwXF3kc+kR6l32Svk0EVCZUEG1Bg+LgVKQ1WG7IhZDOqx6Wq/Hj971QiLBADfd/ZS6UC/aQEEXDta3LJQQFgrVFN52mWbUXtl07RUi6hz2HKjpceW7oXKDBHVluV0jFuWyQ0h4YIcbHjZrtRN10o07y+bnyLIQQoSFEF8HdgBfQrmedgghviqEaNkppvd9FfC98c/VxkIOFSHEVUKIrUKIrf39kw9rNxjmHTViEbYtLO2GcueVWKgLaFGEsAQErFo3VEiNVXW8CRPdhJPXbqjmYhHTg3/yufrzpovaPRWOt1XEIiPi5EoesVCgYikMSXXBtvMDnLyyk2WdUXrFiDpIYnElrtLMDVUsFirn0S7TFEOtuaGIdQMw3L+v+jnUuKHCWiyCVMWiVNY1Fw3cUAA5u001liw0uF/PzV5fKGjuhvoAKk12hZTyFB17WImyRv5xCu/xMuB+KeUB/fMB7V5Cf9d9gtmDqufwWa63Ndo+BinldVLK06SUp/X29k5heQbDPKFcQCIoYauus9qymF9ioSyLYu1IVR8riI1a63jrwirnW3JDJXR77mK2foC7pMUiGk+CrkkYcSMM50pjYhZpoqoTbUo5JZKRIKvD+piJRZOIhbpgC69MruSC55EkSznY4oVYi0W0PMKeEW2daPeZzA8R1e1MQjiV6vyydnmJJsHpvO8Ga1TFnR+atRoLaC4WlwJvl7I6JVw/fhcqnbZV/oyqCwrgZqozvK8EbqrZ/madFXUWkNLuql8CFwshOnVg+2K9zWA4snAKOFYIEGNiFvNLLHQQXtSxEgI2Qd1SfbxYBLQbarLU2URbBwClXP2753I+gycF8VgchMCzI4wSx5NUivIUgnxsGQxXR++si+lgd6Kv6oaqtx4tFiEcck/8EmdoJ5aQlMOtxSyIqxbiXWKUbQe0+0lbFjI7RAJfLMqU9OdU0mJhNREL19YV66U6NSiep1Jn5yjA7UkpJ6xKSpmhRdeRTrN9CWPbmX8ceIkQYhvwYv0zwM+AZ4FnUC6vd+n3GwI+Atyrvz6stxkMRxZOAcdSF2HLEgg/ZjGfsqH8mIUMV2dZ+ARCNZbF2CC35RaUG2oSsUi2dwDgNMiGKhey5AnRHtN3/8FYZZaFckNVLYVC20oY3lH5+ZiIdgUl+iqWRVVcas9DuaF6RYqeH78J8SM1TKgc7mi69goxJRbdYpSnDuh7bb/yPDdIUotFuEYsnJJyQzWzLKRf3e3UaSVSGAHkrLqhmgW4pb6Tr9ccuaU2mFLKLNA9btsgKjtq/L4SeHeD43wV+Gor72kwLFicAmURqtx9WwHfDTWPivJq0nuD4/umB2MEPXUfN77zrO3myROaNGaRiMUoywBugzoLp5AhR5iuuC8WUSKhLshBLGSPqZtwkyth2+8rAeFlwVHKMkAw2knBUcHnZm6oleIAAklgz70AeC2LhbrkLQ/neboiFrqYMD9MUvjzysuMuC4QpFz2LYsm7VB0jMb/HYzB7xk1i26oZmLRDtxHfbGYZ/0HDIYjgHKBstDV21ApyptX2VA1/asq87d9wm2EHBUjGGNZeB62V6RAHWtkHMKyyIkoslDfsnCKWQoyTHdC34Ef+2JsVsEBiIfHWhaycxU4ecgchLY++kSKftqJF13yJR2zsBuLxQqhEmUkAoHEjbR4126HIJxkVSDHr4e0c0aLhVXO0KkvuyHhUiqr321ZWxbN3FBCTyfEqdMYcZZbfUATsZBSrp61dzUYDBNxCpSEHqkKiICfOjufLAu/JUmdauxwkqCjLoqFWstCu03ycnLLAqAgIlDOksqX+fEDe7jirFVYOutKlnLkRZhYSF+6XvU5jnE9rm5/jguP66NWv6zO1erB8A5o66NTDrNPdjA8nKOg3T+RUON2HyuFyr3JnHwVsfuvoxxfOunaK8S66S6lSRe00JeySoTcEmHhULSihL28DmwncEoF/daNxcIK+WIxN5aF6R5rMMwXnAIlv3qbqhvKc+dTzEIPaPLquKHCbQT1QJ8xloVfbFjPdVWHkhXFKmf58QN7uObmx3hkT02wu5SjbI0dR2oHLP7q/LW0x4JjLItQzxr1YEQFudvKgxyUHewZzlcKB5u5oZZry+LAGVdzbvE/8NqWTLr2CvEeOhhVYiGlilm0L688XQipi3pZi4TrWxbBxnUWVki7qMoTYxb3PP6MejBHqbMGg+Fw4hQoiWDFsrAqbqj5aVlMSJ2NJAmUMwjGTcvTrylbkZZmiZcDcWwnW/H317bNsJwcbqCxXz8YsLC1FRLp1WKhM6LChX76ZTt7RvItuaGiokTG7qQkbfbSU/m9tESshzYvRaboKLeRdKG9WgFQiqgguFNU4uvoDreBZmIRbGxZ7NmnqwmMWBgMRwHlAgUZqhSWVdxQ3jyKWWjLIlNPLMJtCCQximNTZ/WdsGs1vhDW4tgxbCfHtoPKSnmyVizcAp7dfNpeNBhACFRX2sRi5YbyXER+kGGrk32pAgXHJWCJ+paOFcAP1Y4EeipW0mRpv2OId5NwlFhIvyCvRiykzpjy3U+uLsprJhaBsD7vOpaFlx3ElQKn1cLBadDKPItPCSE2ztoKDAaDwilUqreh1g01/8Qi5wYnxh/CqsVGgvxYN5S2LJwmFkEtMhgn7OXZ5lsWB6rV3EE3jww2F4tISE3ME0JA5yrlhsoOIKRHPtTDQKZIoewRsa36lo4QFetiwOqupLdOlvY7hlg3MWcY1/Mo5rTYddSIRWIRUBULT4uF3UQsbD2S1quTDSUKKdLEKM3iuPZWzv4J4DohxN1CiL8SQrRYmWIwGKaEU6DARDeUnFdikQMrSMGzCNoTYxYACZGvG+B2A61ZFjKUICrzDOfK2JYY44YKeYVJ53hHghaJsA6Ad65WbqiMaiBRivYymCmRL7v14xU+Wiz2y05KeojRlCyLWA8B6dBGnqzfFLHGsvBiqsuEo0WiFcvCDis3lFMcW/7meRK7lCIl4xNSlmeSVrrOfkVKeTbwZmA18LAQ4ltCiAtmbVUGw9GIUyDvVd1Qlj0f6yxyEIxRdr06qbPKBZIk18CyGBuYboQVjhMTSmDOXdfDQKZEf7pIoewSoYgVbj5SJxoMEA9rIehYBaO7IbUbADe+iKFsSR2rqViojKg9bkflAjwly0JXcXeKNPmstozivZR0Wz0ZH2tZuDpmEQw1FotwKERJBiaIxUC2SEJmSRGv3x5+hmjp7IUQAWCD/hoAHgLeJ4T49qytzGA42igXKMhay0KJxbyyLAopiLRTcmXdmAUoy6JezMJr0Q1lRdqIo+60X7lZpas+tT/NcK5EjCJ2C2JRsSy6jgHpwVM/VcdO9DGULVEse/Xbk/toy+L5ckfFspiaG0pXcTNK0W+KGIqTFkpQRUJZFp7+bKTvhgo1/oyiwQAFQrjj2n3sGynQLrKMytiEyvmZpJWYxWeAp4A/Af5FSnmqlPLfpJSvBE6etZUZDEcbTpGctCvujkAlwD2PLIv8CETbKbseoQluKHUhTJBnNF+T7qsti1bFwo62ERNF2iMW561XF9Un948yOJonLMoEo4mmr1/ZHWdNjxaU41+hgtwP/A8AwY7FDGSKLbuhdpSS9KfVhTwealbDPA5dxd0lRin5MYtQnBESeFhYWkxcX0h9y6KJGyoctCgSwiuNjVnsHcmTJDfrlkUrZ/8w8EHdumM8Z8zwegyGoxcnT96r1gr4AW45n8SikIJIB066nhtKWRY9wSKD2ZopcDpmIYOtiUVIT8s7scemJxGmJxHmqf1pju/SKbGx5mLxmT/dPHZNF38Efvh2CLfT3pak6OxlMFuqP1LVR7uh9skufvrwPpZ1RFne2ZobDYC4LxZpSnklFjIUp9+Ns8ROYIfVZ+EHtqUe4RoMNxaLWMimIEPY48UiVeAUkWXUm+OYBcrldJwQ4pSar7VCCHuag5AMBsN4PA/cEhnXnpANxXwqytNuqHzZnXix1WKxKFRiqFYsKpZFaxfbiB6AdEKPOv6GxW08uT9NOp3Sz7c1fC2oIj271kV20htg1dnQtabSU2rPcL7lAPc9O4Y4b31vSzUiFbTl0EW6MiI27YbZ77VTCndh62ps6buhdAuP4CRuqCLBiuvKZ+9Innb8mMXs3Vi0Yln8J3AKysIQwInAY0C7EOKdUspbZm11BsPRgr77znnBagW3PQ/dUIURiHSQK7rEwvXFoidU5O5MrVjoi1uwNbGI65kWJ/ep429a3s51tz9L/5CK3cRizcViAkLAm74H5QI9u9RnOZApEgk2SewMBHEDqv05wPnrpzgjJxRH2hG6nFG8ohKLA4UAn3Quo+vMZZzqWxZ+nydtWdhN2n1EQxYFQhMaCfYPp4iIMqMyNjYLbYZpxbLYC5ysBwudiopTPItqPf6JWVuZwXA04YtFTYA7oOdZzDc3lIy0kyu7xMa397YCEIzTHSgykKlpdudf3OzWxCIcUxfxS9YpC+P01V04nuT+Z1SVcjQ+jcKzUBzi3RXLAhq0+vAJhHDiiwGBbQleeGx3433rIQTEuugWaTzdFHFvTrBb9hJbdSpB37LQYiF1G3rRpOtsNGhTIIQc16J8dGRAfZ9ly6IVsVgvpXzM/0FK+TiwQUr57KytymA42hgzf1sHuG3lN5832VBuGUoZnFAS15PVZn61hNvoCBTGuqGcPC5W0yZ5Ywipu3lLNyU8ZVUnQsCO/apX02Sps82odKtlErFI9OF2r6+8fzISnPJ7iVgPPVYaWcqAHWF/Rv0e+5KRaopseaxl4cdK6hENBSjIIGJcu49sahCAUTn3Ae7HhRBfBPw02cv0tjAwj5ypBsMCxm+2J0M1lsU8C3AXVApoOajcQPF6g4MiSZKeEgvPk6pbbDlPoYWRqhVCOoCt3Tft0SDH9bUROagvqC26s+rRHa8GkJumzr72i1ByCW+7m5cc3ze9N0suY8X+R+kvZyEUZ39KCUNfMoIoKgtCulWxcLCwrcYCFgup1NlasSg5Hm5uGELMesyild/elajpdX+rv54F3oISClOYZzDMBOWqZVEJcFvzzA1VGFHfbCUWjSyLhMjheJLRgr6XLOcptjD4qHoMLRal6kyL01d3EcOPfTSv4G5GNBSouM/qNhH0ibQTS3bxq/edz1+cvXp6b9a7npXswy6llViM5ulJ6NbuthYtnTIr3BLOJPfuEV1nYdXMszgwWiCJSiCY7QrupqvTxXg/k1JeAHyqzi71J5QYDIapUdPG23ePCO2SkPOlkaAWi7ylxWJ8gBsg3EY0p/YbzJboiKmAbH5KloV2M5Wq2fqnre7kV/cUxz4/TbriIXKlfP2RquNY0TV9YaLnOII4LC5sh84E+1MF+pI6JqHFQlQsizJlgjRLLo6FAhQJYbnVmMWje1KVMa2jxOaugltK6QKe6QdlMMwyzkTLAqG/zxexyI8AkAtUZ15PINxGxFMXr0E/I8rJU5DBaYjFWMsiIg7dDQXQnVAX6qYxi5mgdwMAS8rPa8uiyGJfLCwbD1GJVQivhCOaWxbBgCrKC3i6kM+T/Mevt7EmoSy4lJz71NkM8IgQ4lagIvVSyr+ZtVUZDEcbvljIauoss+WGyg6AHam6e1qloOocMsSBUgM3VJKQ44uFvmsu58nVxGImJaRTY4tVsVjaEeW8VTGVmxk8NMuiW2dETakx4HToWQdAAA9CcQ7sK3Dyyg71nBCUCWJpy8Lyyjhi8iC6a4Wx9WtufmgvT+5P85lTovC4zoaaKzeU5of6y2AwzBblGjeUf1G1ZinA/Y3XwMoz4eX1PMtN0G6otEgAww0ti4CeljegM6LcUp48IdoiLbbLCNgqLlEcHbP5lce3K7GYpOvsZPjps624oQ6JSJIRu5cOpx/XjjGULbEkWXU0lQkiKpZFGbeFy7EbCGN7Siyu/c02TliSZEO7B8EYXik4t9lQUsobhBBRYKWU8qlZW4nBcDRTcUNVx6oi9MVspsUitQsGpjGrWVsWo1JdrBtZFqKYRuAxpN1QXilHQU5BLEAV+BXTY7eV84BQVtEh4KfPNg1wzxCD0dV0pPvVXHGgr71GLEQQS4tFwCvhWpNbFl4gQsBzkW6ZD6Q+RGHdKxCFEYi0Ey5aFMpz20jwlcCDwC/0z1uEEDfP2ooMhqMRneFSO8/Cd0PNqFhIqS7Cer7DlMiPQCBE2lUXtXiDALdAsjjiMZhV5+SV8hSmYlmASp/1xaL/aXj+7kp7dKbSdqMOvhtq1mMWwEh8LQAZqeIki5PjxMKrWhatuKH8Zoz5XJoXiQc5PnV7pV9X2LbmvEX5h1ANA0cApJQPAsfM2ooMhqMRnTtfrBmr6ge4ZzQbqpxT86DT+6f+Wt0XKqf94rFg/dRZgBVxpxrg1tlQifAUCttqLYvffgy+9xaVHXWILiio1lpEQ7M/VTqbVJfKwZI69yU1loUjQhWxCHhlvFbEQltV+aF9WELSk31Gt2BpJ2wH5rzOolynYeAsDu8zzCQDmSK3P90/18swTEaNZREKjI1ZIGfwAuBfgAsjdWc5N6XSF0qJV12fvxaLpdFyxbJQ2VDTcEP52VD5IUjvVe6zQ8yEAuhp02JRT+xmmGLnsQBsH1FFjJXW6YBrBQlosbBkGa8FNxS6c29hSA1zasvvgdQeiHaoFuZzbFk8JoS4HAgIIdYJIa4F7py1FRlmlG/8cSdv/uo97E9N8cJwtCIlXH8xPPL9w/u+un9SkVC1fmE23FCFmqDxVF1RFcvCJRgQ9VNh9UyLJeFyxbIQTmFqAW7/OBVh0/equ+895EwogBcc080/vuIETlvdecjHmoxy9wYcafHQUIBTVnWO6YbrilBFLALSaU0stGXhDO+pbhvari0La85blP81sBEoAjcCo6hKbsMCoD+tROJXT0zDR300khuCXXfDvocO7/tqy6JIsDrlzQ9wz6hlMQNiUXTqB7eh2qY8XKrMtAg4OmYxZTeUXquu76CQmhE3VMi2eOs5ayZO+psFIskeXl36CN8sncsZq8cmFbhWEFuqGglblvGsyXtnCV29Lkf3jnujDiLBOXZDSSlzUsoPSClP151nPyClNLepC4T+tPqHNWLRIiM71ffDPUPCySsfthDV/H/thhIzOYO7ViymGrfIj0C0g1zJrd8XCiCiLIseu8hwroTrugS84tQD3OHERMsCDqnVx1zQFgnymFxDnginTRCLMAGpW5O36IYS2g0l0uPFon3WA9yT/vaEEOuBvwNW1+4vpbxw1lZlmDF8v/GdzwySLTrEw7Pvp13QpHap726x+X4zjVOkbIWJh+3qkB1rFiyLmXBDpdzGNQrasugKFpEShkdH6UE1SExMOXU2o4ZC1QrcAhML30oMBgRbVnSMec6zQthSnZstHUotiEVAi0Ugq4S+3HEMwZFnVczCDpCfxdTZVn573wP+C/gKME86mhlaZTBTYml7hL2pArc/3c/LTloy10ua34w8r777LaMPF+U8ZVHjggIQAg+BmI0AN0zNspCyEuDO9je56dBi0WGpGMxwKk0P4AQiU3P7hNvAK0NuEKSnUmlLmRlxQx1O/N/nicvaJwisFwgR9N1QOBQDk7uhrLA6/3BOCb238oUw8mzFshjJz97fbSu/PUdK+UUp5T1Syvv8r1lbkWFGGcwUuXjjYtqjQX71xMG5Xs78xxcL5zCLhVOkJMITqqJdArMTswjGpiYW5ZzqURVpJ1eqM1LVRwe4k0KJRWp0RL/fFLOY9HFI6d/HyrOq615AJKPKWhgfrwCQVoigVK3cg5SRTWZZ+NhaLCOFg4zKGPYyPW88orKh5npS3k+EEO8SQiwRQnT5X7O2IsMh86MHdvPw7hHyJZdsyWVRMsypqzp5dI8ZmT4pI3PlhsqPDW5rPCzETGZD+ZZF91rITEEs/CBztINcqYllYQUglCCh22aPjur3a3FKXgV/pkVKpYiy6mz1fYGJRXs0yCdev4m3nrtmwnNeIESQMiXXI4QDgXCdI4zFDqvPMV4aIEWCwKoXqESIrmNmvc6iFTfUlfr7/63ZJjGFefOWD938OOes6+H9L1NdL3viYdb1JbhjWz9l1zssWSALloob6nAHuIsUZXBClpEU1szHLIJxSC5T+fktv07faGjLom5fKJ9IO1FX1UhkMsqSCYSmalnoZoK+WCw9GRKLoX351I4zD/jT01bU3S4DYUI4FB2PIE7TKXk+dkSJpYVH2mqDxSfB1c9DOEHYfnhuGwlKKSdKomHeUnY9Uvkyu4dylTz37kSIgNVG2ZXsHMxy7KIpDrw/WpCyGuB2DrNlUVaFa+Pv2GfFDRVJQqIP9kzBm6ybCKqivEnEIpwk7GYQArJZJRoiPEWLoCIWWtBiXfDuuw95lsW8wg4Roky26NCOg2ghZhGqGSmb03NF/O7Bc9buQwjx/2oev2Hcc/8yaysyHBIjOXVH/PxQrpIJ1Z0Is75P/WFtO2DmVTWkMFL16R/uALdTJC/tCf2WpJhpN9SouhC3LVatylud7+27oSLtZEtN6iwAIklEcZSuWIicFgt7qoHpiljsqrwv0Y6W7r4XDNqyyBQdZVnYk4tFMFL9HHOB5JjnwnNYZ/HGmsfvH/fcJa0cXAjRIYT4vhDiSSHEE0KIF+iYx61CiG36e6feVwghPieEeEYI8bAQ4pSa41yp998mhLiy8TsahnMl/b3McwPKb9wdD3HsogRCwNNGLBrju6BgDsRCzXwYb1l4BBByBu8Wi2kVPE70ARKyLSY9aMtCRjvJt+CGopCiOxEin1N/b8HIFC2CSoC7RiyONOwwYUpkCiVCwm3JsgjXiEUxOPYziWjLQko540uF5mIhGjyu93Mj/gP4hZRyA7AZeAK4Gvi1lHId8Gv9M8DLgHX66yrgiwA6mH4NcCaqoeE1vsDMFiXHW7DtMSrN24AHd40Ayg0VDQVY0Rnj6YPpBq80VILb8UVzYlnkPHtCsZsnAjObOlvQbqi2xernVjOi8sMAlELtOJ5sXq8TTkJhlO54mFJeDUKqvSNuifC4AHc42XjfBYoIhAgIWbG+hD15gDsaDlKUyroqhcaKRTgYQEoou4dfLGSDx/V+noAexXoecD2AlLIkpRwBXg3coHe7AXiNfvxq4BtScRfQIYRYArwUuFVKOSSlHAZupUXLZrpc//vnOPcTv+H32wZm821mBd+yALh/pxpQ47sM1vcl2HbAiEVDfMuiZ91hT52V5TxZLzjRshAWQs5g19liWrl4ElosWi3Myw+DsMgLf5ZFM8siCcVRuhIhilosQtEpTuXz3VDZfiUUVpP3W6j4TQGzKnlAtOCGigYDFFFi4YQ6xjznV/7PliuqmVhsFkKMCiHSwCb92P/5pBaOvQboB74mhHhACPEVIUQc6JNS7tP77Af69ONlwK6a1+/W2xptH4MQ4iohxFYhxNb+/kPrsrp3JE/Zlbzjv7fyyO4FlG7qlkmP+AInWZp6gEXx6sVnXV8bzw1kKbumaXBdRp5XKZuJw29ZSKdAUQaJj8+GwpphN9SouvhGO9TPhRb/vvMjKl5RVveJrbihemJBCnl11xyZqhsqGKvOII90TO21CwTfkihlVZzMCk4uFrFQgAJqP2/c51IVi9n5/24oFlLKgJQyKaVsk1La+rH/cytRJhs4BfiilPJk1Pzuq2t3kMq5NiM2k5TyOt276rTe3t5DOtZwrkRvW5i2SJCP/vRxAO7bOcTrvngn9zw3NBPLnXmkhO/8OZf84TJAcnb4Ob4X/jDvEt+r7LK+L0HZlewYyDY+ztHM6G6VmhkIH/46i3KBAhNjFlLMcMyioMXCr1cotfi3kB+GaGelPXnTAHc4CZ5DX4zKCNBIbIqWhRBV6+JIjFcAli8WOZ1e3ELMIhIMUPAvv9Gx3nh/aNZsTcubzYT73cBuKeXd+ufvo8TjgHYvob/7EbY9QG1C8nK9rdH2WWMkV2Z5Z5TXnbqMrTuHSeXK3HDnTu7bOcwbr/sj37r7+ckPcri558vw9C9oL+zhuMgwF8W3A/C63HcrHVTX6ZRZE+RuQG4Yol0qK2UOKriLBCdkQ3kigDVTMQvPhXJWuYn87KRyrrXX5odV2mxJrWVSNxTQFyoSRX2OsfgUxQKqcYojVSyCSiycXEr/PPm42FrLwoqPrY32x/EedsviUJFS7gd2CSGO05suAh4HbqZa6HclcJN+fDPwZp0VdRaQ0u6qXwIXCyE6dWD7Yr1t1hjOleiMhbhwwyJcT/LrJw/w2ycP8vJNSzh+SZJv/HHHbL79BD7208f5wm+fabzD8A649R+h93gAzglv5xSxjX2yi6zdAT98B6T2sLJbXSD2pfKzv+iFSCGl0zNDh9cNJSWWW6BIaKIbSlgzF+D204LDbdW5EKUWxaIwAtFOsqUWLAvtHlkUKhARJcoyQDw2jaFFfhX3ESoWgZASh7IWi0ALbqhojVgEYuPEwndDzVJh3myX8v418E0hxMPAFuBfgI8DLxFCbANerH8G+BnwLPAM8GXgXQBSyiHgI8C9+uvDetusMZIr0xELsmVFJx2xIJ++9WnSRYdLT17GukWJyj/M4eJ/H97Hjx5oYkw9dwc4BXjD18iLKKeIbRxbeoK7vOP5xboPKV/8f51D2/67CQZEZc5AM/7rtu1H34Q93ShPuaEOo1j4syxksDr4SCNFADFTgyn9Vh/hJARsJYrlqbmh8tqyqDt/20dbBN22siym3J68chzthvLjK0cYizrU59Q/OAhAIDh5NlTEroqF3dY95jnfDTUXAe5DRkr5oI4jbJJSvkZKOSylHJRSXiSlXCelfLF/4ddZUO+WUq6VUp4kpdxac5yvSimP1V9fm801Q9WyCFiC89f3sns4TzwU4Oxje4iHbbLFWW6+O7oXPr0R9j+i0nhHCzzbn6n8o05gaDtYQehex1OB9Zxd/iOJ8iAPeMeSWXYevOM2sCOI2/+dzliIoUzzC+Foocz3fvlbvnP7YR4ANNfoeQ0EghPF4r6vw47fz877+vO36/SGkiJAYKayoXR78lu25/jlY/tV3KJVy0J/NtkpuKE6AnkivlhMpzX+ER6zSLYpy2loWN37tiIWliUoCyUWkbYGlsVCc0MtVIqOS67k0hlTQaQLjlsEwIuOW0QkGCARtskUZ9my2HO/CrZu/w37UnmkBE/CU43SXge3Q+dqCNjcL9fT4ao7lfu9dXQnQioVdPlpMLqXrnhoUsvi3ueGuCH4r7xoz3WzVuAz73AdKKWVZWFry6L23H/zUbj3+tl5b33HnyZapzfUzFsW33kkxWdufVq1zmglZuF5FTdUviU3lLq4d5AjIop6/vY0Kq+PcLHws6HCrrLuWnFDAZRFmFEZJREd69oLB33LwojFYcFvl9ERU7+4C45bxKruGJedrmLs8bBNyfFmN/106Fn1fd/D7Bmuxhce3zvaeP/utUgp+WNJ9XeUdpSNJ7+QF67tUfskl8HoProToTG1GPW4a9sBljJIj7Ofg+nDnBU0V/gppH7MAqrNBKVUbhhdmDbj6OOmZKKOZTGDqbM6ZjFQDvPk/jTlQKS1bKjiqJopEe2sWNWT9YYCiMosSVEgS+TQ3FBHqFgQU26kJULd3NmhyQPcACUrxrBsmzBMqhqzmB3PhxmbNg7/QtqpxaI9FuS2/3tB5Xk/tTFbdCqCMvOLeE593/8Iu7VYWAIe31cnJ97zlFisOZ982eXu8loIgFh6Mv/2p6dW90sugVKaJeEy9400/2N6cvt2LCHpESke3ZOiL9naH/GCptIor73aRNAtqsyoYlrNcphlsRiWiboxi8CMBbiVZZFB3ZGmnBA9rVgWtU0Eh1uxLJRYiOIoq62DPO/2cFwzcWnEkS4WHasAWCvUiNRgC9lQADdG/pTc8AE+Pc5aiyzUbKiFStWyqG82J/Q/86y6onzLYnAb+weHsQRsWdFR17K45Z4HlSuh+xgGMyVGSfDsitfBKVeM3bFtKQCrQikGM42thcFMkdRBVQPZI0Z5dE8Da+ZIo+aCiN92wbcsfJHIz1JehT7+CIk62VABrJlyQ2nraVTG6G0Lc7Botxaz8M8/2kmu5BIMCEJ2k0tHKKEK6vIjLOcg+wJLqqNip0JFLDqm/tqFQCSJG+lkraXEwg5NHrMAGIiu4W55/AQrtBrgNmJxWBjRlkUjsahaFrNj6r37m/eT3rcNwu0gPbz9j9KXjLB5RQdP7EvjelU/eq7k8LWbf6V+6FpbsYq2v+BfYcvlYw+cVONUlwVGGC04Dd1odz07xCIxAkCvSPHYnlm6m55v1Az3qXQ29S2MiliMzNJ7q+MX7DYC1riLqo5ZzEjsSFsWMpTgT09bzsFCAKfQQvuXilh0sGs4T3t0kviDEMoVNbSdKAX67aXTW++RblkAVtdqlgp1ExJsUSz8KYUTxWLu2n0clQxry6KzgYvJF4vZsCwKZZdbH91FLL8PNvyJer+hx1neGeWEJUnyZZcdg1Uf857hPKuFagT3eLGXIR247orXWXubEos+/Yc53CDIfdezg6yw1R2ojcvze/bOzMnNd2otC39imZ8R5V8si6OzMxRJH98NTbwoSitAAA9vJvIMiqO4WCzp6ebCDX1kZYhCrhWxGAFgVCT45WP7ueTExZO/JpKEvQ8C0Ll8/fTWexSIhdCuKGgtGwogEgoQDwUm3FhUK7iNZXFYGB+zGE+iJmYx0zw3kGUp/QTwGFx0FkTa6c0+xbKOKCcsVX7gx2pcUbu1WBRlkM/em2suFkl1d9ejM6WGGgS5H9w1wont1aC6kz7Q1G1VyzMH01z4qd9xML0AO/aOsSz8APc4sajdb0bfe5iSCBMIT+yfJEUAGxfHm4ELQCFFhhhrFyVY3B4hTwRRG7PwPPjlB6D/6QnrA/jptgIlx+PPzlg5+XtF2mFkJwBvecWF01vvqrNh/SXQdQQP5excXX3cQrsPgFgwMCG4DbUV3MayOCyM5MqEbYvo+IDcU7+Ajy3h1G8cxyeD/zUrYrG9P8NqobqA3j3Sjtd3EqvL21neGWPdojZCAWvMHO3dwznWiP2MRJZxyxP93PSgsgK66gldMArRTtodVWjn11pIKXn1F/7AD+/fTaHs8sS+UdZGq+1AekWKJ/a11qn2N08e5Nn+LE/vP4ztRJ79HfzbmkO/iNcGuP3un+PdUOMfzxT5YTJWW/223zpm4c6AaeGOHuCg184xvSrrKifDBNyaav5sP/zx8/DETWNfqD+b/34wxeYVHWxc2sKdftjfR0BHC+JSj551cPl3qq1JjkQ6q5ZFq2LxwmOVZTieUGBhV3AvOIazpfpWxdbrIdxGuWcD51sPzYob6pmDVbH4+d4Y2a4T2CCeZ3lHmJBtcfzSJA/pGRUAu0fyrLEO0LXyeDYvb+e2p/sJWIJktEGmSttSEiUlFn6txWjB4aFdI3x36y4e3zeK40mWWimwVcZML6nKxL3xjL+APaQ79E6WmjujHHxSBZ6HdxzacfIjyECYwaI1wQ1VzAzW7DcLQe78CBnRNmGWBQCWRQAPZwbEopDaz4BsZ21vgngoQI4wtlMVC1fHLwb27x63vmG8QITH+0tcfkb9edIT0BlRtC+vJgwYJjINy+LNL1jNv146sfG3ZQlCgdkbrWrEYhzDutXHGLID8MyvYfOf4Wx4Nb0iRSk7MuPvvb0/ywnRQUpWlF/udNknFhERZVbF1MV6y/J2HtmTqlyk9wxlWSUOEOw9lhv+8gw2LG5jaUekceZJcgnhvBIj32Xlf9+6Y5g7n1HtzTu9IejbCECPSDFaGCuMzxzMcOVX72HLP99Cf00dht/OfSQ/C379RvgtLFqdy9CIQopR4rz5q/dUA9xaLDJDNdPkZsmySJGob1lYNgE83BkYaCPTB+innWN649gBi7IVwZYl1WAQGEmpcxs4sGvsC/PD5G118T93XYsdnf04Q9eaQ173EU1HrWVx6CNjw0HLuKEOFyO5OpbF4z8G6cJJbyDYuw6AYGrHtN/jl4/tr9u59pmDGdYH+ym3r6bkSP77EeX7XxFSbp1Ny1XXz+396ufc4B5ClKFzNR2xED981wv57jte0PiNk0uxsyog7ouEH49wPMnX/rCDRW1hgrkDsGgD0grSI1KkC9WLf6bo8Nov/IG7nxskXXS4Y5uyVIazJZ4fUv7vkRZ6T80YpZkSixGG3BiP7xsl6+k7fC0W+dEBPKkE2MsONjrC9MkPM0K8fr8lEZgxyyJYGGRAtrOmR8VGXHtsm3J/CE8gN64nWH6EjEjQFrZZ0t5izY3fMbbTiEVT2ldQGTw6AxZY2A4Yy+JwMZIvc5H3B/jEWuULv+GVqv137/HQt5HgomMBiIw+N+G1H/zxI/zrz56YtLr7v/+4k3+86dExcyU8T/Jsf4blcj+xxcfywZcfz46i6h3jp7JuXtEBVMelWr5gdawGVKHUkvYm3T3bliIyB+mJiqpY1FzYB7MltixvU77rtiUQ72WRlSJdY1nsTxVIFx3+5bUn0RUPVaYJPlwTS/Ezyg4LRR0fOUSxKGWGGPRiSAnPDOjPRLcpL2eH2CNVJXw2NQvTE/PDDHvxCTUWAFh1YhaeB/d/o/VZFAClHGE3SzHSQ0SnXnr+TAsd5C5psYgWxwlifoRBL876xW2t10v4bihjWTTHDilXHbTshmpG2LZMzOJwMZIrcWrhLlWxu/E1qu9S/5Ow6Q0gBKLrGDwE8ezOMa+TUvK9rbv50u3PcvmX7xpzNz6edKGM60k+86un2ZfK840/7uDZgSy2k6WrsAux6ATedu4xfOatLwUglFd3esf0xGkL2zy8e4R8ySWZ12mttUGyZiSXAJJjY9kay0J994XoBX2eau3QthiR6GWxNTrmXFJ5tX93IswL13bz+2cGkFLyyO4RAI6PjlRqVRpRdFyu+sZW7tvZxKXzxy/A9RdPfk6+Gyp9aGJRTA+RkuqO+8l+v4Jbn0dumN2yF0da5EZmoRNvfphBL97ADVUnG2rXXXDzX8N9N0zcvxFZ5UqzEouq2yptypXolPLqs0y6w2PqOmR+iAOlCOv7pjCTwndDGcticjpWqSLGGRgda9xQhwkpJSO5MstKz8KKM+EVn4G/eRCu+DG84D1qp2CUg/TQnhsrFtmSS9HxeMEx3dy7Y5jvbt094fg+6aKDEHDzQ3t58adu459ueoz3fvsBTraeUdW6K84EoGexDibqu2bLEpy0vJ2Hd6fYM5JjhdAXrvYWg466ivuY0GglaD2kv7/+FDWpdkuHTntNLIb4InrHWRbDWV3hHg1y7roeDqaLPH0gw0O7U7yu4xl+Lt9FONVk9gZw385hbnn8AB+6+bHGxWZ7H4Bdd0PmYP3nfUozY1nI/AijxFjUFuaxg/oz0NPygqUR0lYbIyQopmfYsijnwSnQ78Qau6HEOMtiz/3q+9M/b/19MupvJdxRrZGQQW2FasvC9cVC5BgYqaZoe7kR+p0Y6/vaWn8/E7Nonc7VM2JVAHzlzafxwZefMCPHGo8RixrSRQfhlenJ76gEeLFDsPaCMf7EvfYyugpjxWBAB3pff+pyTliS5CcPNS5mSxccXnbiYha1hTllVSevP3U5j+0d5VTxNBIBy09XO/pDamoumJuWd/DEvlGeOZhhhdVPKba4Mvh9UnStxcrgSMWyGMiUaAvbXHb6Sj572RY2+2LRtgQSfXQzViz84HVHLMg5Otj5nXt38cDzw7w4oVxzoew+mnH3syqj6JE9KdUqux5+IFkXdjVC6rbbk4rKJNilFG6onXOO7eGR/TpDSBfgRZw0oUQ3KRnHyc5wNlROF0nKeP1+S9oN5ccsyq5H5rl7AfB23El/f2vnnR9Wf4/x7ur4essfLqRbfriFasrz3j36ZsjzID/ICAmOm4pYbHgFvPhD0Dcxa8cwji2Xw9l/OyOHOqZX1dDMBkYsahjJllkr9mJJpyoWdTgYXMai0lix8O/UuxMhXrl5KQ/uGmHXUP2+O+lCmeWdMf549UX891vP5GOvPZETliR5QegZRN/Gqr8XILEI0tUL6ktOWETZlXz850+yQhwcm00xGR3KAlltHWBIWwhD2RLdiRAh2+I1Jy/Dyuo79LY+SPTS4aXI5KsZT5V2KIEiy/b8kmN64nz1D8+RLbqcHtEDmgrN+0nd/dwgGxa3sbY3zqdueRqvXvDW78S6/R5STbKrRkd19fNoc4FqiucR8bJE2rrYsrKDA1m9HqdIsezQJtPEO3pIiQQiN8Ni4feFkgmSddpoCMvGrolZXP7lu+h/6o/skd1Y0uFfr/0Cu4cn7+80dFD9vXb1La9ss8J+gFuJhFfT+mNgn86IGniKgFvkKW8F6xdPQSxiXXDO/wHLXGImZfXZcMH753oVk2J+kzUM50psEDpLqe/EhvsNhpeTkOnKXSGoO/QEOXpiAV6xSbXW+Okj4y5g2QHcO7+ALBdoC9tYulw/bAf41ltP4wx7e8UFVSHRN8bFcuqqLl65eSk7BpUbKtizuvUTjLRDx0pWlbcznCvheZLBbHFsxbcvTPFFEF+EjYOsKXhL5ctYAtoe/DJ870r+8Sybt5+7htv+74voyTwJqG6j43l87yj/8attFMoujz1/kC+4H+HvThhl28FMpbPuGPRF9L67fsPHf/5Ew1OSlQD3wbHzJ6ZANj2MhSTZ2cuWFR2U/WbMbpHdBwcJC4doew8Fu4NAqU7n30OhpongMT0TK7grloVOnd1/YD9rrAO4W95MKdTBeWzlC79V89b3pfINi/eyg8qyWLy0KhZ2RFsW2g0lS1XLIjWghX/XPQBsj5xAT8LUSxzNGLGoYUVXjHefUEAGQtB9bMP9RqL6bn6w6pt39z7CH8J/wzF3/QMrumJsWdHBjx/Yg1ObGXXntQRu+Qe+Gvx3OuyxQeCOzDNY5QysPGvsm7X1TfDH/8OfbKA9JFkshhC1RT2tsHgTS/LbcD3JaKHMYKZEd+1FYO8DSqDskLJqgFChmh0zkivTHg1ibVNj0C9o388HXn4Ci4IFNb4VCDmjEy5a//rzJ/jMr57mgz9+lHXus6xN38Pm/T9Qx8xPDIi7OXUR3cj2phXkdlld4AJuvhq/mCLP7lJ33d09fWxYnMSx/DqLMvv3q4tsonMRTqidSHl2xCIl46ztnRhAFro3lOtJpJSsKW0DYOWm8wltuISLgw/z/a07+defPcELP/4bfnBf/VhZKXWAIZlgVW9HZVtFLPzOs8Xq51cY1jc6u+5hVCSJ9K07xBM1LHSMWNTQFQ+xTu5E9G5QM4obkI77YqHu6Bh6jvPveQftIkf08e/B8A7+/KxVPLk/zTv++z5y/szup3+Bk1jKmdYTvPixcWbn83ep75NYFgBL2qN8/uW9WMipuaEAFm+iPfc8MQo8P5Qjncnwwf3vhd99XI0NffoXcPrb9XsrsQgXq0Hd4VyJNZEs7LlPbTjw6NjvQILcGNfRMwfT3LFtgLBt8f37dnOipWIbi/bfhoU3MdXW8xC6SK5PjDDa/3zDQHjIzdEvtdtumhlRj29XIrdsyWJCtkU0ogO/TpH+g8rS6u5ehIx2EfdmuGW7FotSqIO+ZJ07d8smoLOhCmWPjej29Uu3wJLNxNxR2kWeL93+LFKqqv66ZA4wJDrHdCoNRZVYODpWYZWzHJCdALj6s/R23c1Wbx3HLU5iOLoxYjGeA481dUEBlNpWkCcEz92mNvzkvQivxNv5IMIKwJ3X8vpTl/OR15zIb586yEf+93EYeg76n6T/xLfxWed1LDl4m2pVASpT5a7/VD10xvfRSSxScwjKY5vzndujc+xbTZv1WbIJgeR4sZMn96c5u3Abq3KPwu/+Fb59ucqseqHO/OpaC8Bq97lKXCGVL3OB9aB6PpyE/Vok9j0MgCdskuTGtPy44c6dXBK8n7tXfIFIwOPsuHJx2MVhThFPT0i1PdDfj4XHvi4lnMeUttFfr5mh5xLy8jwnldtvKhlRe0fyHBwtIKXk7ieU6Hd0qYB9RSzcMildvZ3oXIQd7yROAac0g40StVh09yyqW8MQCKgK7kLZI10oc5L1LOnYCoh2VjKO/vmly/nIqzfSEQs2TFsO5gfIBcfNbNZiUcpXxWLUaiMXSGLn+pHZQazBbdzrrOOlG1voNGs4ojFiUUt2ADL7oa956lk0EuZG9yLkw9+Fh78Lz93GLzr/nO2J02HzG+GB/4HUHq44axVnHdPN0wcysO0WAPb3vYgb3QvxrJDqN1XMwLf+FEb3weuuV7MAaknof9LxF8Jhna0yDcsCYLO9k63PDXKl9XOG4sfCljcpUXrJh1XTQYD2ZaSiKzlbPEpGW0cjuTIvdO+F5HI47mVVi2L/I5DooxhbTFLkKhetdKHMD+7fzXuTd9Cx7w6+/hKhxGL5GUjL5sWBByoDp3xuuU/FKHpOeglSWGyytrP9YJ0CNO12es6bulh8+svX809f+Cq/e6qf/KiOPekhO4loGA8BbhFXZz+JWBehpCrM6+8/xGrxWvLDlLBZ1ttT9+lgUInFaKHMaMHheLGT0Y7j1ZNRtd6XHxvlihespisWqmS5jSfuDFGOjn2PUEwFrMs6sG27WYoiRinSQ7s3wp5H7wBgX/IkXrC2+1DP1LDAMWJRy4BuzdwkEwrUTIv/Kr8CLBt+9A6IdXOT/VK6EyGVAics+O/XQvoAXXH9D/zUz6FnPf2hZQzSTmrNy+HBG1WF+L4H4fVfhRVnTHyzhO4uOT41dGQnWMFKOmzLJJdCrJszo3vJP3M7G62d7Fz3ZnjV5+Hd98CJl47Z/WDvWZxpPUE6q9wb2VyWEwv3w/qXKgssvQ+yg0osFp8E4SRt5Cr1GE8fSEMpy3F5VRtwVv73JFLbYM25yFXn8GLrvjFiIaXkjoeVX757yWpKS8/g9YHb2XGgTpsN7WN/TipB9dNDpZQ8czDdMNj7zME0705/jg8WPsW7v7mVjbZOc9afdTIWokwQ3BLC70Yb7STWriyPwX7lmnrgD7/gka13tPa5N6CcHWJEJljbIC01HAxi4ZHKl0kXynSJNDKu/yb8WgY9AU9ZFhMzx4qOS6c3XP1b0iRiUYrSppxXQhx0cpQCMaKdi1kSGOXWX96EIy1OPuvC6U26MxxRGLGoZdUL4e93qj76TUiEbQ7SSeGkN6lq5xf+NXtylsoW6V4Lb/o+pHbB50/nIzsu538yb1Muq/UvrXSrLWz5C9XX6MBjcNk3K8OOJr6ZrrjNHFA573d9ET53impB0r586lWfQsDikziZJ3hH/nqGZILccZeqFMfe4ybsPrrkbBKigLNrKwDh/H7CMg/LToXF2l331M+g/wlYsgUR7SApqm6ooWyZc6xHsLwyRLvg/htUdfySzVjrX8qx1l7c1J7K+z24a4TcqBaGaCfBC/+BpWKIrsf/Z+K56OrtfaKXkgyQ6t+NlJJ//snjvPjTt3PeJ37L/9y1c8LLfvvAU6yxDrBcDHCG+wBvCt8Gx1wACSUGyUhQiYVTIlAcqaylq0ddbHfv3YO7aysn3PrnyFv+cWqf/ziyI/2MNAhuA4RCYWw8RvNl0vkSbeQRsQ715DixqNyYjGPPgQHioki4faxYxMM2ecK4WnRDXo5yIEa4YwknxNO8xL2dB1jPa84wwW2DEYuJRDsmbejl9/DpP+VvlSVxxlUMZorKsgCVN/3mm+H4V7Kv4xTucjfgbX4TnP62SoFbaPWZcPHH4C3/21goANq0G2rkeeWu+sXVatvxr4QLPjC9c1y8iUWlXRwj9vH/ld9JZ3vj+QTF5WfjSYG943Yc1yPkXzxj3dWCq19crayc0/6SQKydJNnKHe5wtsSF1gN4oTY4528raZos2VwRp+Bo9YK+ezhPBzorJ9qJtfZ8HrQ384K9N0ys39BuqM7ObgZopzC0l5v++7Osv+eD3Nr1Cf6r/EFKP//ghHN6/tE/VB5fm/g6neWDcPpbK9uS0SAlAuCWCJVSlEQIglEWL1EFbWsf/ATut/6MMGXanEOruyhlVMHbsYvqpM0C4aCNJSSpXIl8JoUlJMGY/n35s6m19dMRC9WNWQzsVzUTsZqCPFA3PTnCeEVlWYTdPI4dg/giwtk9LBcD2Of/3eRjVA1HBY1TfgwN8Xv4jAY64CX/jOOqjJ7ueI3IrDgdVpzOfXft5B9//CjnXngRi5IR0gXlYklEg9VAcjNiPYCA2z+hgqEv+3c44+0TYxtTYeNrGHnqDq7Y+zoekcfw8USo4a7R9h4ek6tYvuf3pPJlOoW+kMe61Z24n6113v+D9mXYsQ6SIl9jWRS4NPAg3tqLsDa8Am79J13vsapSIR3OVC2L0UKZDlEVC4BfLf0r/s/z74av/4mywvygvrYs+np66U91snH3/7Kam8iF24h2n0D64AGOdbZRdNzKyMldQzmSQ49CEDj5z2l74H9Utfr6l1XWkIzaFKWNdIpEyiMUgklCgOg7kdtWvofVO7+HG8hyt7eBYxlXS1PKVd2ZoFySiUXKqhJiYhvq3DCjJDm5u75YCJ2Vl84XKOogfzCuA9X1LIvxYnHLBzn2idvV7h1jLYu2iE1ehono3lARmce141VrdsVZnHzB6+uuy3D0YSyLaTB+tKr/D9pT56LbrQve/O6u6YJDyLYqF69JCdgQ71FCcepb4MyrDk0oAJadSvpNP+MRqcZVNhohC9AWCfJHbyPtAw8wkq2564/pC9bSU5RgnP1eAESkg3aRq6TDBgaeZpEYIbD+JcpFt+gE1c5EiEq3zXih2hollS+TRAezdQA3uPJ03lr+O+TwTvjan1TmL/hiEYonGQotxZWC/+l8F+F/eB7x1lvYvuoyoqLEyMhI5fi/efIgm63tlDuOgRf+DSDU51qTKp2MBClJG7dcIOqmKdr6omwF6H7p/+NFxU9xSu5a7vY20CFHkV5N47afvg+uO7/69aVz4VPHwUd74SM9cNsnxny+dimFE2onGGjwr6jdjOlcgXJWZU6FE0pECSWUGNXELAplj3xJr8ctw53XEs4f4Hb3JIIrTh1z6Li2LCjlQEpi5JGhRDUj78IPHPrfmuGIwVgW08Bv+JbVGUIDaV8sJrqv/Opov7truuiQrDM/tyntK1SG0sUfne6SJ7CsI0o0GCAYEITsxvcMyYjN83IRlnTIDB2gS+gCOX3Xz6s+p8aPhrXPPZIkRp5UVqWXSl0RLvzuo2/6fvXuOhglFeiio1i9O0/lyyyysshgDKHdgcf0xvmMu4UDp/wfFv/xn5Vwxnso50cJAqFYktuP/X98adcBvnTVawjoC6/dpmIQqcF99PWobJ69qTyXWM9ir3iJcoO943bo3TDmnNujQUoEyRcKdIgMTrjqptu4NMmKrgTPD+Ww4j0EipLMSD+JLuUuTO3dxn65ki8G/gzbEqRzBV7Y5/K6E+JEH7oBa/dWai+/UWcU2dbZ+BcltFjkizihEfURJ7RQW5ayLnSFvT9OdzhXIhqKVjoMbF1+JVc+upknu8amvybCNs8TobucBbeEjYsMxeGEV6vPZrHp62SoYsRiGviWRaao7uCqfaEmioVvbfj7pAsObZEp+oBff70a9RmeQm+eSbAswfq+xJgmgfVoiwQZ1EVvxZF9dIg0UlgI319e2/IaINKOhaSgJwnKnA5W+5ZI+1i/+Wh4MT25airqaN5ho51HRKsX0HW6NfbzpQSLQaU4x3soZlNKLOLt/OMlW3A8OUb4/IBudmg/oILxcnQ/fWIYlp2idlqyacI5J6NBytgUCnnayeJGqmsWQvCKTUv48h3PctzatfA4pAf3VcQiN3yA3dZyrn7v+1jUFuab9zzPh3/yGNfsk3w9uIh13vNUjuYUiVLADXdMWEMFbVlk8yU8LRZWtKZALtJeY1lUxWJpRxT0Zz9EkpBtVeZY+MRDyg1lObmK8IpQQom5EQrDOIxYTIP4ODeUbzV013FDdek4xlC2WnfQNlXLouuY6S61KVedt7Zpkz6ASNBiWHQAUBw5QCcZvHA7gUYN4vSENFeLhfBnVke76u6ejS6jL/MwniexLMFovkx3IFu1XIB1i9poi9g8PGRzBkBOVZSXcirgHYm3q/nD1liXSbRTCVkxVU077h55RD1YekrDc05GbErYFLVl4UbH3vn/zUXruPSUZYw+XoTHITtcFbu4m8KLba50/rzirFWcvKKD257uJ//7LsLFaqW7bxHISBPLwtJ/a4UCMqwD/JGahIRIeyXA3RkLslbsIfTQN2HJOyuf04CXqBuktixByYoQcFLkM0p4RWTmbkgMRxYmZjENxovFgA489sQnWhYd0SCWqBULZ0zLhbnk5ZuWcPmZK5vuI4QgH1IXei9zUAW4Y00KtPSFzMmru91gaURtj9UXi2JiOUvFAGndHyqVL9MhstVMHyBgCc5Y3cVd+/Wfa3ZAv8coRWkTj9UPDrd1qWK9crpGLLK6n1eTO+f2aJAiQcqlAh1ksOJj1x4JBjh2URvhdmVNFFJaLDyXhExTDo/d/8Rl7bz7gmMh0Ue7O1xpeCgrQjq5GyqTL2EVdV+qms+GSEfFsjj28Wv5Rehq1t39fhjYVvmcDjhtDV2fJSuK7ebJZdQxbCMWhgYYsZgGibCNEDCq78oHMiWCAUEyOvEf0rIEnbFQTYB7GpbFHFOK6MrfTD8dIo3Vgli4uRGKjkuoNELBijccRu8llxMSLukBld45WiiTJFMJbvuceUwXDw3rY2TVIB83P0qWSMPPM6ldQ16m2tsqVhogY7VBKNbwFJLRIGUZwCqOEhFl7Hh9oWvTx3d8McoNYSFxIvX3l/FFBHEqLT7yup7EijezLNS/aK5QIuB3843UcUOl9tC99TNsl7pIM7234obaW443TH917ChBN08xq44diBqxMNTHiMU0CFiCvrYIe0ZUEPdgukBPItywyrUrHmJIu6oy04lZzDFWJEmZIIF8P71WBtHASgAqF7I2suwayhN3UxSDjes4RKeybHL9OwBlWbTJzIS77TPWdDOMDqLri6BXSJOV0YafpxVpo0gQkauKRbw8SNpu3roiGVEB7kRZvU+4rf7+7d1ajNJKvKQWMdlATAPaEimOqIB+Xs/zDiaafJ7aDeW5ZUQxRUFExgqvH+BOKbH9tnuB2p4+UPmc9pUidWdlAHh2jJAsUNQuvaARC0MDjFhMk+Wd0crQmd1DeVZ0Nr5T7YqHxgW4F5Zl0RYNMmJ1EiwMajdUM7FQwtBGjkf3pOgkQznc+M452K2ypJzBHYCy1mJueoJYnLg0STgUJhdoq7hXZDFNhmjjz1MIUqKdYKFaOJd0RyputUYkozZlbDo8ZQFEkvX7NrXFIgzJBEJflIujes51vP7+ET3SdLhf1ZX4I1pDifr7q3NQbqgAHhRGKQTGVXpHO5RlMaLE4qmgzuxK71OfU6SDoYIk2UBQlViUcHIjAIRjprusoT5GLKaJEgvVL+n5oRwruhqLRXdCuaE8T5IpObTNk5hFq7RFggyLJNHSIO117vrHoNNMkyLHQ7tHdIC48cU52qsL7EZUG/JiIUdIFie4oeyAxamru1Rmlr6Dt0oZMkQaXggBMoEOwiUlFlJKOuUIxXBzyyJsB3BEkIBQsYVQov7+lqXEyC6oi35+RImFn7I7nni3chFl9GChckaJTCMxUm+ixMISHjGZpTheLCLt4ORhSHXOHY6toWDF1BCrnMoaG82XG7qhCvqzCA4+pc493tgKNBzdGLGYJiu6YuwfLZAtOuwfLbCqu7llMZQtkSk5SMmCc0O1RWwOuEmShb2EKU4S4FZ3ph1Wnod3p+gijWgiFh3tHQzIJIHR3WSKDglvbPV2Laes7GCfk8DVMQirnCEro5W6l3rkQ53EyiMA5EouPYxM6L5aDxGoyWxrIo6jgQ4iWowKOusqlFxUd9/ORaoIsaDdUG52CEdaxJvVWWg3lI1Hkizl4Dg3kR/sPvAoRDsJx9sZsroqloWMdTNacOrG0wD6E8oSaT+oJuJF48ayMNRnVsVCCLFDCPGIEOJBIcRWva1LCHGrEGKb/t6ptwshxOeEEM8IIR4WQpxSc5wr9f7bhBBXzuaaW2V5ZxTXk9yzQ10oVjazLOJhRnJlUrqqeaG5oZKRIHudJMulHrnazA0VCEIwxpJwkcf2pugQGQJNfPLJiM1u2UMku4fRgjOh1UctyzqiDMkkbkZZFraTIW/FsBtVPwOlcBdtnsr0GU2nSIgCXrz+xXwMtf3Bmpxv3u4gqsXI0bGLaHt9y6Knp4+itHF19pTMDZMiTnuTCnqEOjcLj6TI4YTGXcx9sdj/KLSvoCsWpJ8u1YIlN4Qb6cL1ZEPLIt2+njIBuocfUmtv62i8FsNRzeGwLC6QUm6RUp6mf74a+LWUch3wa/0zwMuAdfrrKuCLoMQFuAY4EzgDuMYXmLlkuY5R/HG7ciWsbGJZ+PUXu4ZUjGOhWRYdsSADJLGFHhHbxFIAINLO4nARt1yiTeQJNXDLgHIv7bMWk8w9TypXpr3S6mPir3hxe4RBmawErP2W2s1wI910yhSuJ8kNqTt60YJYiFqxaGJZFMLdtLkqtuFlBxiVMZKJ+qm8yWiQQdohqywQURhmRCYa3vUDFTdUAI8kObzwODeRX3Mx/By0r6AzFuKA16Esi9wARZ3G28hVF47GeVquIOCVKMsA8Vjzz9Nw9DIXbqhXAzfoxzcAr6nZ/g2puAvoEEIsAV4K3CqlHJJSDgO3Apcc5jVPYHmnGhB053Z14WpmWfgtP54bVBfCxAKzLC4/cyUXnlYzPbCZZQEQaafLLlb6SEWSjcUCYE9wFZ2lvaRHUywVOnNp3OwFgL5khEHasAvD4HmE3Bxlu35r7wrxHmKiSGo0RXFYWUZ2cuKxx2PZ6ndWRllKjSiHu1X2lusgsgMMyraGd/FCCFKBLoJ5ZYEEiiOkiDe/eagJcLeJ3MQq/trYTscKOuMhdrvtOmYxSN5WzzdaU0c0yMOuSjLIESEcXFh/m4bDx2yLhQRuEULcJ4S4Sm/rk1L6zYD2A/5/7jJgV81rd+ttjbaPQQhxlRBiqxBia39//0yeQ12WtEcRAh7bO0osFKg0DKyHLxY7B33LYmH9Qy5qi3D82rXVDZNZFuEkHSJHp+4jFWyQeupzMKIq1L3+JznB2okXCEP3sRP260tGGJJJBB7kh4jKnOqS2oSAnlExOriP8qgSi1DH5CNCA0FlWeTttqbN9GSsGwuJzA0SKAwxRJL2WOOLfzbYRaykrNFgKUVaJAlYjY/vxywCuCTJVdus+NRWc7cvpzMWZI/TDk4BPIeMboLYKHX2/ON6eVRqsRCRxuswHPXMtlicI6U8BeViercQ4rzaJ6WUEiUoh4yU8jop5WlSytN6e5vfyc4EIdticTKClMqqaDZJzG9dfvdzKr4x5UaC84HaHlDNAtwAkXbayNJZmUvRXFyG4koYAgNPslHswOneULeILxmxVVt4qIyV9ULNLYuQjh9kBvfj6tTWSMeS5usHAiH1O2tWIwIgtIstnzpIsKjEolm2WynSW5mBES6nyAUmCShrN9SicImgcLH8wUc+Y8RCWRYHZNVtlrbU840siw2Lk6S7VDV7QRgXlKExsyoWUso9+vtB4EeomMMB7V5Cf/d7MewBVtS8fLne1mj7nOO7opq5oPz9ehIhHto1giWq/aIWFPEaAW6WOgsQSRL1snT4HWonEZdQ7zEUCBIdfpKN1s6GrTiEENVjPX8nAPlI8/hDVM9wKKQOVGIFia7JxcIOqrvs8iRiYSd8MdpHpDRM1ko2vXHw4ovokCmk6xB10hTsycRCCc+KoBLeYLxj7PO1lkb7ClZ3x8eIxQjq+M3Sized+gJKMkDRijZfi+GoZtbEQggRF0K0+Y+Bi4FHgZsBP6PpSuAm/fhm4M06K+osIKXdVb8ELhZCdOrA9sV625zjB7mbpc2C6iV11/sv4lfvO4+b33NOxS21oPCDwqE2sCdZf7yXUP4gvUL3MpokxnH8sk6e8ZbRu/8OOkUGe9mWhvta+uLMQ9/BQ7C3q87c8hraulRtQ2n0IIHcQYZlgrb45BfFYFiJhTve7TOOiN8famQfMSdFLthcSK22RQSEZHRgNzGZpRyapK6hZz0AZ1hPABAa33okGFEdiQE6VnDS8nYOUF3DkBaLZtPuXnHyGh6Ta0gH5jxvxDCPmU1/SB/wI32XZQPfklL+QghxL/BdIcRbgZ3An+r9fwb8CfAMkAP+AkBKOSSE+Ahwr97vw1LKQ5tlOUO0almAyvo5dtECbqUQ7VTB1lgLF5QVZyDu/i/Ot3WH1UncUBuXJnlKLufE0u8BsJZubrhvpL0PBoADj/CoXIs1vkX6OJJ6braXGSBUGGSIdjqbpNr6hLQbimYdYYGoHlVq7b4XG4fCJNXhYe0CS+16jHbAadaeHNSAqLalnJG7Ty2nXmqr3x8q1kPSskh0L8P3AB5024BM06SKxe0RPr/2Y7THw5zefDWGo5hZEwsp5bPAhP96KeUgcFGd7RJ4d4NjfRX46kyv8VDxxaJZ9fYRg2UpV9RkwW2AVecAcEHgYTwRwWrStA9gw+I2bpHK0+ghsBad0HDfRGc1k+m37qZJq+HDsQ5K0obcAJHSIENWa3fPobD63VqTWEXJzh5+627mvO3fBZjQcXY8cW3pFPc+BoA3ieWCELDyTHof+xEAkbY6x492qOFTuunghpV9pB+P0UaOg06ctnCheRAd+OiVc55gaJjnmAruQ+CFa3s4d10PJ684Ssz39uWQXDr5fm190HMctlfEik8SDAdiIZtUmwpy77GWVafu1aGnPc6IVBlQt7mbG2b5VBCCQauL5Og24qVB0nYLYgcs71Xuoc7e5mm2PW1hbnAvJuCp3l9etPn59qzZRFHadGz7odowWfwHYMWZlYd2PcsuuWzMtL8tKzrY73Xi2VEGS/bkn5HB0AJGLA6BFV0x/vutZzZNlTyiuPQ6eNknJt8PYM256nsrlghgL94IwK7wxJTZWvzCvLyd5EF5LJtXdEx67Ee6L2FT/h66nANkg62tp6NNuQxjzfo2oQLHOztewIGgzuZu0M7cp2fRYn5tnU1vWsUgApPsD4wRizHZTz6v+wq8+guVHzcv7+CA7KAY6lQt341YGGYAIxaG1uleCx0rJt8PYLUWi8kK+DRLVh7L7e5JPJo8v+l+fckIv/FO5kbvYpKxMCctm7zx3fBJf0lOhrFxKYQn7wsFVFN3W7jz37Syixvcl+FIC9m2fNL97+u7tPK40ayMMSw+qVoYGK6TPRXvGfM5b1jSxs/l2dzf8VJS+TLtzSrEDYYWMWJhmB1Wq7hFq2JxwtJ23lx+P9t7X9x0v8XJCB9z/pwP5y7l3HW9k/riATYcs6Yy56GVJoJANcOoBbHYsqKD/8xdwPnFzxBsn7zgL7L6TB71VgMQmqRgUa0lCMtOVWsKTl44F7YDPLrkNXy8+HqGc+WmabMGQ6sYsTDMDvEeOPkKWHdxS7ufsFTXA0xyF9zbVq1ROW9daxf+DUva+Kp8JVu99Qx0Ns60GsPik2DlC5qOX/XZsqIDEOyht2mKqs/GZR18ynkDv3U3E0u2IBYAW94EJ7yqtX2By05bwSN7UjxzMNPSmgyGyTD2qWH2ePXnW951UVuEd75oLRef0DygHAkG6IwFGc6VOX99a5X6YTtA95LVvH73h3hvR/OYSIX2ZfCXv2hp1xOWJgkFLEqu11L86oQlSX7rncxvvZP5TrTFmpstf6a+WuTyM1eycyjLl2571sQsDDOCEQvDvOHvL9kw+U7A4vYofckIi5Kt9zLavKKDh3anZuUuO2wHOH5pkod2jbR0/JVdMRJhm0zRmdUL+dWXbGBpe5Szj23RejEYmmDEwrDg+MirNxKyp+ZB3by8A9g5axfnk1d0tCwWliU4YUmSe3YMzapYCCG48oWrZ+34hqMLE7MwLDhOW93FpuUdU3rNWWu7SUZsjuubnSr6CzcsoiMWZGlHa/2VKjGahdhU0nBUIlTh9JHFaaedJrdu3TrXyzAcZUgpmzYRrOWZg2l+8eh+3nPhullelcHQOkKI+2oG1Y3B3NYYDDNEq0IBcOyiNt5z4QLuFWY46jBuKIPBYDBMihELg8FgMEyKEQuDwWAwTIoRC4PBYDBMihELg8FgMEyKEQuDwWAwTIoRC4PBYDBMihELg8FgMEzKEVnBLYToB3bO9TqmQA8wMNeLmGGOpHM6ks7F50g6pyPpXHzm6pxWSSnrtnM+IsVioSGE2NqoxH6hciSd05F0Lj5H0jkdSefiMx/PybihDAaDwTApRiwMBoPBMClGLOYH1831AmaBI+mcjqRz8TmSzulIOhefeXdOJmZhMBgMhkkxloXBYDAYJsWIhcFgMBgmxYjFNBBCrBBC/FYI8bgQ4jEhxHv19i4hxK1CiG36e6fevkEI8UchRFEI8Xc1xzlOCPFgzdeoEOJvG7znJUKIp4QQzwghrq7Zfr0Q4iEhxMNCiO8LIRIL/Zxqnv+cECKzkM9FCPF1IcRzNcfYMtXzmYfnJIQQHxNCPC2EeEII8TcL+FzuqHn9XiHEj6dyLvP0nC4SQtyvX/97IcSx0zmnCUgpzdcUv4AlwCn6cRvwNHAC8Angar39auDf9ONFwOnAx4C/a3DMALAfVRRT77ntwDFACHgIOEE/l6zZ79P++y/kc9LPnwb8N5BZyOcCfB14/RH2N/cXwDcAy3+vhXou4/b7AfDmI+D38zRwvH78LuDrh/r3J6U0lsV0kFLuk1Lerx+ngSeAZcCrgRv0bjcAr9H7HJRS3guUmxz2ImC7lLJe5fkZwDNSymellCXg2/q9kFKOgrrbA6LAtDIW5tM5CSECwL8D/2+hn8tMMc/O6Z3Ah6WUnv9eC/hcABBCJIELgR9P5Vzm6TlJIKkftwN7p3NO4zFicYgIIVYDJwN3A31Syn36qf1A3xQO9UbgxgbPLQN21fy8W2/z1/A1/X4bgGun8J51mQfn9B7g5pr3nTbz4FwAPiaUm/AzQojwFN6zLvPgnNYClwkhtgohfi6EWDeF9xzDPDgXn9cAv/Zvvg6FeXBObwN+JoTYDVwBfHwK79kQIxaHgFDxgR8Afzv+j0wqG7Clu3whRAh4FfC96axDSvkXwFLU3cxl0zlGzVrm9JyEEEuBNzAzojcffj/vR4n46UAX8PfTOEbtWubDOYWBglTtKL4MfHUax5gv5+LzZzS+MLfMPDmn/wP8iZRyOfA1lHv6kDFiMU2EEEHUH8U3pZQ/1JsPCCGW6OeXAK2a5y8D7pdSHtCvXVET4PorYA+womb/5XpbBSmlizJFX7fAz+lk4FjgGSHEDiAmhHhmgZ6L756QUsoi6h/3jKmey3w7J9RdrP/+PwI2LeBzQQjRg/q9/HSq5zHfzkkI0QtsllLerbd/B3jhoZyXjz0TBzna0PGB64EnpJS1qn0zcCXK7LsSuKnFQ465q5FS7gK21LyfDawTQqxB/ZG8Ebhcr2OtlPIZ/fhVwJML+ZyklI8Bi2v2y0gpp5TNMV/ORT+3REq5T6/pNcCjUzmX+XhOKL/+BcBzwPmogOpCPReA1wP/K6UsTOU8aplH5zQMtAsh1kspnwZegvI4HDqtRMHN14RMhHNQ5uTDwIP660+AbuDXwDbgV0CX3n8x6m5sFBjRj5P6uTgwCLRP8p5/gvqn3A58QG+zgD8Aj6AuQt+kJjtqIZ5TnX2mkw01b84F+E3N7+d/gMRC//0AHai78EeAP6LuZBfkuejnfgdcciRcE/T21+rfzUP63I45lHPzv0y7D4PBYDBMiolZGAwGg2FSjFgYDAaDYVKMWBgMBoNhUoxYGAwGg2FSjFgYDAaDYVKMWBgMM4AQwtUFU48J1QX4/xNCNP3/EkKsFkJc3mwfg2G+YMTCYJgZ8lLKLVLKjahCqJcB10zymtWMLQ4zGOYtps7CYJgBdKV5oubnY4B7gR5gFardelw//R4p5Z1CiLuA41GV0DcAn0NV+r4I1X/pC1LKLx22kzAYmmDEwmCYAcaLhd42AhwHpAFPSlnQHVpvlFKeJoR4EWqWwSv0/lehZkN8VHen/QPwBinlc4fxVAyGupjeUAbD7BMEPi/UlDwXWN9gv4uBTUKI1+uf24F1KMvDYJhTjFgYDLOAdkO5qC6j1wAHgM2oOGGjhnUC+Gsp5S8PyyINhilgAtwGwwyj20T/F/B5qfy87cA+qSbLXYEaiQnKPdVW89JfAu/Ura4RQqwXQsQxGOYBxrIwGGaGqBDiQZTLyUEFtP1W1f8J/EAI8WbgF0BWb38YcIUQD6Fmdf8HKkPqft3yuh89htNgmGtMgNtgMBgMk2LcUAaDwWCYFCMWBoPBYJgUIxYGg8FgmBQjFgaDwWCYFCMWBoPBYJgUIxYGg8FgmBQjFgaDwWCYlP8fKQ/5tLd7xBAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.figure()\n",
    "plt.plot(multi_X_test[\"timeStamp\"], multi_y_test, label=\"Actual Demand\")\n",
    "plt.plot(multi_X_test[\"timeStamp\"], multi_y_pred, label=\"FLAML Forecast\")\n",
    "plt.xlabel(\"Date\")\n",
    "plt.ylabel(\"Energy Demand\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Forecasting Discrete Values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Dataset and Preprocess\n",
    "\n",
    "Import [sales data](https://hcrystalball.readthedocs.io/en/v0.1.7/api/hcrystalball.utils.get_sales_data.html) from hcrystalball. The task is to predict whether daily sales will be above mean sales for thirty days into the future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hcrystalball.utils import get_sales_data\n",
    "time_horizon = 30\n",
    "df = get_sales_data(n_dates=180, n_assortments=1, n_states=1, n_stores=1)\n",
    "df = df[[\"Sales\", \"Open\", \"Promo\", \"Promo2\"]]\n",
    "# feature engineering - create a discrete value column\n",
    "# 1 denotes above mean and 0 denotes below mean\n",
    "import numpy as np\n",
    "df[\"above_mean_sales\"] = np.where(df[\"Sales\"] > df[\"Sales\"].mean(), 1, 0)\n",
    "df.reset_index(inplace=True)\n",
    "# train-test split\n",
    "discrete_train_df = df[:-time_horizon]\n",
    "discrete_test_df = df[-time_horizon:]\n",
    "discrete_X_train, discrete_X_test = (\n",
    "    discrete_train_df[[\"Date\", \"Open\", \"Promo\", \"Promo2\"]],\n",
    "    discrete_test_df[[\"Date\", \"Open\", \"Promo\", \"Promo2\"]],\n",
    ")\n",
    "discrete_y_train, discrete_y_test = discrete_train_df[\"above_mean_sales\"], discrete_test_df[\"above_mean_sales\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Open</th>\n",
       "      <th>Promo</th>\n",
       "      <th>Promo2</th>\n",
       "      <th>above_mean_sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-02-02</td>\n",
       "      <td>24894</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-02-03</td>\n",
       "      <td>22139</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-02-04</td>\n",
       "      <td>20452</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-02-05</td>\n",
       "      <td>20977</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-02-06</td>\n",
       "      <td>19151</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>2015-06-27</td>\n",
       "      <td>13108</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>2015-06-28</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>2015-06-29</td>\n",
       "      <td>28456</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>2015-06-30</td>\n",
       "      <td>27140</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>2015-07-01</td>\n",
       "      <td>24957</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>150 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Date  Sales   Open  Promo  Promo2  above_mean_sales\n",
       "0   2015-02-02  24894   True   True   False                 1\n",
       "1   2015-02-03  22139   True   True   False                 1\n",
       "2   2015-02-04  20452   True   True   False                 1\n",
       "3   2015-02-05  20977   True   True   False                 1\n",
       "4   2015-02-06  19151   True   True   False                 1\n",
       "..         ...    ...    ...    ...     ...               ...\n",
       "145 2015-06-27  13108   True  False   False                 0\n",
       "146 2015-06-28      0  False  False   False                 0\n",
       "147 2015-06-29  28456   True   True   False                 1\n",
       "148 2015-06-30  27140   True   True   False                 1\n",
       "149 2015-07-01  24957   True   True   False                 1\n",
       "\n",
       "[150 rows x 6 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "discrete_train_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run FLAML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from flaml import AutoML\n",
    "automl = AutoML()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "settings = {\n",
    "    \"time_budget\": 15,  # total running time in seconds\n",
    "    \"metric\": \"accuracy\",  # primary metric\n",
    "    \"task\": \"ts_forecast_classification\",  # task type\n",
    "    \"log_file_name\": \"sales_classification_forecast.log\",  # flaml log file\n",
    "    \"eval_method\": \"holdout\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[flaml.automl: 11-07 01:56:17] {2600} INFO - task = ts_forecast_classification\n",
      "[flaml.automl: 11-07 01:56:17] {2602} INFO - Data split method: time\n",
      "[flaml.automl: 11-07 01:56:17] {2605} INFO - Evaluation method: holdout\n",
      "[flaml.automl: 11-07 01:56:17] {2727} INFO - Minimizing error metric: 1-accuracy\n",
      "[flaml.automl: 11-07 01:56:17] {2869} INFO - List of ML learners in AutoML Run: ['lgbm', 'rf', 'xgboost', 'extra_tree', 'xgb_limitdepth']\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 0, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3297} INFO - Estimated sufficient time budget=76s. Estimated necessary time budget=0s.\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.0s,\testimator lgbm's best error=0.2667,\tbest estimator lgbm's best error=0.2667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 1, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.0s,\testimator lgbm's best error=0.2667,\tbest estimator lgbm's best error=0.2667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 2, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.0s,\testimator lgbm's best error=0.1333,\tbest estimator lgbm's best error=0.1333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 3, current learner rf\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.1s,\testimator rf's best error=0.1333,\tbest estimator lgbm's best error=0.1333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 4, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.1s,\testimator xgboost's best error=0.1333,\tbest estimator lgbm's best error=0.1333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 5, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.1s,\testimator extra_tree's best error=0.1333,\tbest estimator lgbm's best error=0.1333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 6, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.1s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 7, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.1s,\testimator lgbm's best error=0.1333,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 8, current learner rf\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator rf's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 9, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator lgbm's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 10, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator lgbm's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 11, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator lgbm's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 12, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator xgboost's best error=0.1333,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 13, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator xgboost's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 14, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.2s,\testimator extra_tree's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 15, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.3s,\testimator xgboost's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 16, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 17, current learner rf\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.3s,\testimator rf's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 18, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 19, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.3s,\testimator lgbm's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 20, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.3s,\testimator extra_tree's best error=0.0667,\tbest estimator xgb_limitdepth's best error=0.0667\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 21, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 22, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 23, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator lgbm's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 24, current learner rf\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 25, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 26, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 27, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 28, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 29, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 30, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 31, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator lgbm's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 32, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator lgbm's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 33, current learner rf\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 34, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 35, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 36, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 37, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 38, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 39, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.6s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 40, current learner rf\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.6s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 41, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 42, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 43, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 44, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.7s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 45, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 46, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.8s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 47, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 48, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 49, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 50, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 51, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 52, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 53, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 54, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 0.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 55, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 1.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 56, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 1.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 57, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 1.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 58, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 1.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 59, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:17] {3344} INFO -  at 1.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:17] {3164} INFO - iteration 60, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 61, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 62, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 63, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 64, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.1s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 65, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 66, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.1s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 67, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.1s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 68, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.2s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 69, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 70, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 71, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.2s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 72, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 73, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 74, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 75, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 76, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 77, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 78, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 79, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.3s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 80, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 81, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.4s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 82, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.4s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 83, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.4s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 84, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.5s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 85, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 86, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 87, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.5s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 88, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 89, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 90, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 91, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 92, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 93, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 94, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 95, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 96, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 97, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 98, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 99, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 100, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 101, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 102, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 103, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 104, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 105, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 106, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 107, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 108, current learner rf\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 109, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 110, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 111, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 112, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.9s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 113, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.9s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 114, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.9s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 115, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 116, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 117, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 1.9s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 118, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 2.0s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 119, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:18] {3344} INFO -  at 2.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:18] {3164} INFO - iteration 120, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.0s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 121, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.0s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 122, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.0s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 123, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 124, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 125, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 126, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 127, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 128, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 129, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 130, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 131, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 132, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.1s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 133, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 134, current learner rf\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 135, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 136, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 137, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 138, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 139, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 140, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 141, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 142, current learner rf\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.3s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 143, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 144, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.3s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 145, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 146, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 147, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 148, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 149, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 150, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 151, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 152, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 153, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 154, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 155, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 156, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 157, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 158, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 159, current learner rf\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 160, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.5s,\testimator extra_tree's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 161, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 162, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 163, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 164, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 165, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 166, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 167, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 168, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 169, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 170, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 171, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 172, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 173, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 174, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 175, current learner rf\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.7s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 176, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 177, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 178, current learner rf\n",
      "[flaml.automl: 11-07 01:56:19] {3344} INFO -  at 2.8s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:19] {3164} INFO - iteration 179, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 180, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.0s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 181, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 182, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 183, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 184, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.1s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 185, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 186, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 187, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 188, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 189, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.2s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 190, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 191, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 192, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.2s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 193, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 194, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 195, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.3s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 196, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 197, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 198, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 199, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 200, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 201, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 202, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 203, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 204, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 205, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 206, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 207, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 208, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 209, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 210, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 211, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 212, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 213, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 214, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 215, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 216, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 217, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 218, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 219, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 220, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 221, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 222, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.9s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 223, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.9s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 224, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 225, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 226, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.9s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 227, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 3.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 228, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 4.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 229, current learner rf\n",
      "[flaml.automl: 11-07 01:56:20] {3344} INFO -  at 4.0s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:20] {3164} INFO - iteration 230, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.0s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 231, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 232, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.0s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 233, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.0s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 234, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.0s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 235, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 236, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 237, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 238, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.1s,\testimator rf's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 239, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 240, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 241, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 242, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 243, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 244, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.2s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 245, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 246, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 247, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 248, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 249, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 250, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 251, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 252, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.4s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 253, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 254, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.4s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 255, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 256, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 257, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 258, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 259, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 260, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 261, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 262, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 263, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 264, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 265, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 266, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 267, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 268, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 269, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.7s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 270, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 271, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 272, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 273, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 274, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 4.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 275, current learner rf\n",
      "[flaml.automl: 11-07 01:56:21] {3344} INFO -  at 5.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:21] {3164} INFO - iteration 276, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 277, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 278, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 279, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.1s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 280, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 281, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 282, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.1s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 283, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 284, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.2s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 285, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 286, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 287, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 288, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 289, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 290, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 291, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 292, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 293, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 294, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 295, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 296, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 297, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 298, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 299, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 300, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 301, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 302, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 303, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.5s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 304, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 305, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 306, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.6s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 307, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 308, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 309, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 310, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 311, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 312, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 313, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 314, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 315, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 316, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 317, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 318, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 319, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 320, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 321, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 322, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 323, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 324, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.8s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 325, current learner rf\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 326, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 327, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 328, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 329, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 330, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 5.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 331, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 6.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 332, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:22] {3344} INFO -  at 6.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:22] {3164} INFO - iteration 333, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 334, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 335, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 336, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 337, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 338, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 339, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 340, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 341, current learner rf\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 342, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 343, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgb_limitdepth's best error=0.0667,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 344, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 345, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 346, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 347, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 348, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 349, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 350, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 351, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 352, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 353, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 354, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 355, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 356, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 357, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 358, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 359, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 360, current learner rf\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 361, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 362, current learner rf\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 363, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 364, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 365, current learner rf\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 366, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 367, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 368, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 369, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 370, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 371, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 372, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 373, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 374, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 375, current learner rf\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 376, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 377, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 378, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 379, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 6.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 380, current learner rf\n",
      "[flaml.automl: 11-07 01:56:23] {3344} INFO -  at 7.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:23] {3164} INFO - iteration 381, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.0s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 382, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 383, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 384, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 385, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 386, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 387, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 388, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 389, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 390, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 391, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 392, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 393, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 394, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 395, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 396, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 397, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 398, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 399, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 400, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 401, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 402, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 403, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 404, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 405, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 406, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 407, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 408, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 409, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 410, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 411, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 412, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 413, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 414, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 415, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 416, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 417, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 418, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 419, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 420, current learner rf\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 421, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 422, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 423, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 7.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 424, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 8.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 425, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 8.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 426, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:24] {3344} INFO -  at 8.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:24] {3164} INFO - iteration 427, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 428, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 429, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 430, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 431, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 432, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 433, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.1s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 434, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 435, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 436, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 437, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 438, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 439, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 440, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 441, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 442, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 443, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 444, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 445, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 446, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 447, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 448, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 449, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 450, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 451, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 452, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 453, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 454, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 455, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 456, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 457, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 458, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 459, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 460, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 461, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 462, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 463, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 464, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 465, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 466, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 467, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 468, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 469, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 470, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 471, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 472, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 473, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 474, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 475, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 476, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 477, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 478, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 479, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 480, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 481, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 482, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 483, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 484, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 8.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 485, current learner rf\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 9.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 486, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:25] {3344} INFO -  at 9.0s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:25] {3164} INFO - iteration 487, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 488, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 489, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 490, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 491, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.1s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 492, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 493, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 494, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 495, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 496, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 497, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 498, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 499, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 500, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 501, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 502, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 503, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 504, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 505, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 506, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 507, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.4s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 508, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 509, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 510, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 511, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 512, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 513, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 514, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 515, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 516, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 517, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 518, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 519, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 520, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 521, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 522, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 523, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 524, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 525, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 526, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 527, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 528, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 529, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 530, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 531, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 532, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 533, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 534, current learner rf\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 535, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 536, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 537, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 538, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 539, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 540, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 541, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 9.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 542, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 10.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 543, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:26] {3344} INFO -  at 10.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:26] {3164} INFO - iteration 544, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 545, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 546, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.0s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 547, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 548, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 549, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 550, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 551, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 552, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 553, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 554, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 555, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 556, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 557, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 558, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 559, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 560, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 561, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 562, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 563, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 564, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 565, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 566, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 567, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 568, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 569, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 570, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 571, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 572, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 573, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 574, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 575, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 576, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 577, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 578, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 579, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 580, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 581, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 582, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 583, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 584, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 585, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 586, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 587, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 588, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 589, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 590, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 591, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 592, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 593, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 594, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 595, current learner rf\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 596, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 597, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 10.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 598, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 11.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 599, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 11.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 600, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 11.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 601, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:27] {3344} INFO -  at 11.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:27] {3164} INFO - iteration 602, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 603, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 604, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 605, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 606, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 607, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 608, current learner rf\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 609, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 610, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 611, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 612, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 613, current learner rf\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 614, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 615, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 616, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 617, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 618, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 619, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 620, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 621, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 622, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 623, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 624, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 625, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 626, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 627, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 628, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 629, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 630, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 631, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.5s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 632, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 633, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 634, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 635, current learner rf\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 636, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 637, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 638, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 639, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 640, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 641, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 642, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 643, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.7s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 644, current learner rf\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 645, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 646, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 647, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 648, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 649, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 650, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 651, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 652, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 653, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 654, current learner rf\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 655, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 656, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 657, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 658, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 11.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 659, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 12.0s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 660, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:28] {3344} INFO -  at 12.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:28] {3164} INFO - iteration 661, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 662, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 663, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.0s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 664, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 665, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 666, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 667, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 668, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 669, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.1s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 670, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 671, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 672, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 673, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 674, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 675, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 676, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 677, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 678, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 679, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 680, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 681, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 682, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 683, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 684, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 685, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 686, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 687, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 688, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 689, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 690, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 691, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 692, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 693, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 694, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 695, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 696, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 697, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 698, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 699, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 700, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 701, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 702, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 703, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.6s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 704, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 705, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 706, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 707, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 708, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 709, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 710, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 711, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.8s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 712, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.8s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 713, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 714, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 715, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.9s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 716, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 12.9s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 717, current learner rf\n",
      "[flaml.automl: 11-07 01:56:29] {3344} INFO -  at 13.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:29] {3164} INFO - iteration 718, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.0s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 719, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.0s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 720, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.0s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 721, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 722, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 723, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.1s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 724, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.1s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 725, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.1s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 726, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 727, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.2s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 728, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 729, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 730, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.2s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 731, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.2s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 732, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.2s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 733, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 734, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 735, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.3s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 736, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.3s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 737, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.3s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 738, current learner rf\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 739, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 740, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 741, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 742, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 743, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 744, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 745, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 746, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 747, current learner rf\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 748, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 749, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.5s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 750, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 751, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 752, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.5s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 753, current learner rf\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 754, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 755, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgboost's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 756, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 757, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 758, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0333\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 759, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 760, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 761, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.6s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 762, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 763, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 764, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 765, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 766, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.7s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 767, current learner rf\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 768, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 769, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 770, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 771, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 772, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 773, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 774, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 775, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 776, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 777, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.9s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 778, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 779, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 780, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 13.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 781, current learner rf\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 14.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 782, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:30] {3344} INFO -  at 14.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:30] {3164} INFO - iteration 783, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.0s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 784, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.0s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 785, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 786, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 787, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 788, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 789, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 790, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 791, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.1s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 792, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.2s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 793, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.2s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 794, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.2s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 795, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.2s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 796, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.2s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 797, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.3s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 798, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.3s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 799, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.3s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 800, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.3s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 801, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 802, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 803, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 804, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 805, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 806, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 807, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.4s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 808, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 809, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.5s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 810, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.5s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 811, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.5s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 812, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.5s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 813, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.5s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 814, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.6s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 815, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.6s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 816, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.6s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 817, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.6s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 818, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.6s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 819, current learner extra_tree\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator extra_tree's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 820, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 821, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 822, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 823, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 824, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 825, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 826, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.7s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 827, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 828, current learner rf\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator rf's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 829, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 830, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 831, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 832, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 833, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.8s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 834, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 835, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 836, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 837, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 838, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 839, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 840, current learner xgb_limitdepth\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgb_limitdepth's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 841, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 842, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 843, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 14.9s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 844, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 15.0s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 845, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 15.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 846, current learner xgboost\n",
      "[flaml.automl: 11-07 01:56:31] {3344} INFO -  at 15.0s,\testimator xgboost's best error=0.0000,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:31] {3164} INFO - iteration 847, current learner lgbm\n",
      "[flaml.automl: 11-07 01:56:32] {3344} INFO -  at 15.0s,\testimator lgbm's best error=0.0333,\tbest estimator xgboost's best error=0.0000\n",
      "[flaml.automl: 11-07 01:56:32] {3608} INFO - retrain xgboost for 0.0s\n",
      "[flaml.automl: 11-07 01:56:32] {3615} INFO - retrained model: XGBClassifier(base_score=0.5, booster='gbtree', callbacks=None,\n",
      "              colsample_bylevel=0.8487386958719925, colsample_bynode=1,\n",
      "              colsample_bytree=1.0, early_stopping_rounds=None,\n",
      "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
      "              gamma=0, gpu_id=-1, grow_policy='lossguide', importance_type=None,\n",
      "              interaction_constraints='', learning_rate=0.47977588153251416,\n",
      "              max_bin=256, max_cat_threshold=64, max_cat_to_onehot=4,\n",
      "              max_delta_step=0, max_depth=0, max_leaves=4,\n",
      "              min_child_weight=0.24154961266982103, missing=nan,\n",
      "              monotone_constraints='()', n_estimators=5, n_jobs=-1,\n",
      "              num_parallel_tree=1, objective='binary:logistic',\n",
      "              predictor='auto', ...)\n",
      "[flaml.automl: 11-07 01:56:32] {2900} INFO - fit succeeded\n",
      "[flaml.automl: 11-07 01:56:32] {2901} INFO - Time taken to find the best model: 13.628411293029785\n",
      "[flaml.automl: 11-07 01:56:32] {2912} WARNING - Time taken to find the best model is 91% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.\n"
     ]
    }
   ],
   "source": [
    "\"\"\"The main flaml automl API\"\"\"\n",
    "automl.fit(X_train=discrete_X_train,\n",
    "           y_train=discrete_y_train,\n",
    "           **settings,\n",
    "           period=time_horizon)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best Model and Metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best ML leaner: xgboost\n",
      "Best hyperparmeter config: {'n_estimators': 5, 'max_leaves': 4, 'min_child_weight': 0.24154961266982103, 'learning_rate': 0.47977588153251416, 'subsample': 0.9582292262719722, 'colsample_bylevel': 0.8487386958719925, 'colsample_bytree': 1.0, 'reg_alpha': 0.02723388128976539, 'reg_lambda': 0.0779137867635275, 'optimize_for_horizon': False, 'lags': 7}\n",
      "Best mape on validation data: 0.0\n",
      "Training duration of best run: 0.005982637405395508s\n",
      "XGBClassifier(base_score=0.5, booster='gbtree', callbacks=None,\n",
      "              colsample_bylevel=0.8487386958719925, colsample_bynode=1,\n",
      "              colsample_bytree=1.0, early_stopping_rounds=None,\n",
      "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
      "              gamma=0, gpu_id=-1, grow_policy='lossguide', importance_type=None,\n",
      "              interaction_constraints='', learning_rate=0.47977588153251416,\n",
      "              max_bin=256, max_cat_threshold=64, max_cat_to_onehot=4,\n",
      "              max_delta_step=0, max_depth=0, max_leaves=4,\n",
      "              min_child_weight=0.24154961266982103, missing=nan,\n",
      "              monotone_constraints='()', n_estimators=5, n_jobs=-1,\n",
      "              num_parallel_tree=1, objective='binary:logistic',\n",
      "              predictor='auto', ...)\n"
     ]
    }
   ],
   "source": [
    "\"\"\" retrieve best config and best learner\"\"\"\n",
    "print(\"Best ML leaner:\", automl.best_estimator)\n",
    "print(\"Best hyperparmeter config:\", automl.best_config)\n",
    "print(f\"Best mape on validation data: {automl.best_loss}\")\n",
    "print(f\"Training duration of best run: {automl.best_config_train_time}s\")\n",
    "print(automl.model.estimator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted label [1 1 0 0 1 1 1 1 1 0 0 1 1 1 1 1 0 0 1 1 1 1 1 0 0 1 1 1 1 1]\n",
      "True label 150    1\n",
      "151    1\n",
      "152    0\n",
      "153    0\n",
      "154    1\n",
      "155    1\n",
      "156    1\n",
      "157    1\n",
      "158    1\n",
      "159    0\n",
      "160    0\n",
      "161    1\n",
      "162    1\n",
      "163    1\n",
      "164    1\n",
      "165    1\n",
      "166    0\n",
      "167    0\n",
      "168    1\n",
      "169    1\n",
      "170    1\n",
      "171    1\n",
      "172    1\n",
      "173    0\n",
      "174    0\n",
      "175    1\n",
      "176    1\n",
      "177    1\n",
      "178    1\n",
      "179    1\n",
      "Name: above_mean_sales, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "\"\"\" compute predictions of testing dataset \"\"\"\n",
    "discrete_y_pred = automl.predict(discrete_X_test)\n",
    "print(\"Predicted label\", discrete_y_pred)\n",
    "print(\"True label\", discrete_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy = 1.0\n"
     ]
    }
   ],
   "source": [
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print(\"accuracy\", \"=\", 1 - sklearn_metric_loss_score(\"accuracy\", discrete_y_test, discrete_y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Forecast Problems with Panel Datasets (Multiple Time Series)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load data and preprocess\n",
    "\n",
    "Import Stallion & Co.'s beverage sales data from pytorch-forecasting, orginally from Kaggle. The dataset contains about 21,000 monthly historic sales record as well as additional information about the sales price, the location of the agency, special days such as holidays, and volume sold in the entire industry. There are thousands of unique wholesaler-SKU/products combinations, each representing an individual time series. The task is to provide a six month forecast of demand at SKU level for each wholesaler."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_stalliion_data():\n",
    "    from pytorch_forecasting.data.examples import get_stallion_data\n",
    "\n",
    "    data = get_stallion_data()\n",
    "    # add time index\n",
    "    data[\"time_idx\"] = data[\"date\"].dt.year * 12 + data[\"date\"].dt.month\n",
    "    data[\"time_idx\"] -= data[\"time_idx\"].min()\n",
    "    # add additional features\n",
    "    data[\"month\"] = data.date.dt.month.astype(str).astype(\n",
    "        \"category\"\n",
    "    )  # categories have be strings\n",
    "    data[\"log_volume\"] = np.log(data.volume + 1e-8)\n",
    "    data[\"avg_volume_by_sku\"] = data.groupby(\n",
    "        [\"time_idx\", \"sku\"], observed=True\n",
    "    ).volume.transform(\"mean\")\n",
    "    data[\"avg_volume_by_agency\"] = data.groupby(\n",
    "        [\"time_idx\", \"agency\"], observed=True\n",
    "    ).volume.transform(\"mean\")\n",
    "    # we want to encode special days as one variable and thus need to first reverse one-hot encoding\n",
    "    special_days = [\n",
    "        \"easter_day\",\n",
    "        \"good_friday\",\n",
    "        \"new_year\",\n",
    "        \"christmas\",\n",
    "        \"labor_day\",\n",
    "        \"independence_day\",\n",
    "        \"revolution_day_memorial\",\n",
    "        \"regional_games\",\n",
    "        \"beer_capital\",\n",
    "        \"music_fest\",\n",
    "    ]\n",
    "    data[special_days] = (\n",
    "        data[special_days]\n",
    "        .apply(lambda x: x.map({0: \"-\", 1: x.name}))\n",
    "        .astype(\"category\")\n",
    "    )\n",
    "    return data, special_days"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "data, special_days = get_stalliion_data()\n",
    "time_horizon = 6  # predict six months\n",
    "# make time steps first column\n",
    "data[\"time_idx\"] = data[\"date\"].dt.year * 12 + data[\"date\"].dt.month\n",
    "data[\"time_idx\"] -= data[\"time_idx\"].min()\n",
    "training_cutoff = data[\"time_idx\"].max() - time_horizon\n",
    "ts_col = data.pop(\"date\")\n",
    "data.insert(0, \"date\", ts_col)\n",
    "# FLAML assumes input is not sorted, but we sort here for comparison purposes with y_test\n",
    "data = data.sort_values([\"agency\", \"sku\", \"date\"])\n",
    "X_train = data[lambda x: x.time_idx <= training_cutoff]\n",
    "X_test = data[lambda x: x.time_idx > training_cutoff]\n",
    "y_train = X_train.pop(\"volume\")\n",
    "y_test = X_test.pop(\"volume\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>agency</th>\n",
       "      <th>sku</th>\n",
       "      <th>industry_volume</th>\n",
       "      <th>soda_volume</th>\n",
       "      <th>avg_max_temp</th>\n",
       "      <th>price_regular</th>\n",
       "      <th>price_actual</th>\n",
       "      <th>discount</th>\n",
       "      <th>avg_population_2017</th>\n",
       "      <th>...</th>\n",
       "      <th>football_gold_cup</th>\n",
       "      <th>beer_capital</th>\n",
       "      <th>music_fest</th>\n",
       "      <th>discount_in_percent</th>\n",
       "      <th>timeseries</th>\n",
       "      <th>time_idx</th>\n",
       "      <th>month</th>\n",
       "      <th>log_volume</th>\n",
       "      <th>avg_volume_by_sku</th>\n",
       "      <th>avg_volume_by_agency</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>Agency_01</td>\n",
       "      <td>SKU_01</td>\n",
       "      <td>492612703</td>\n",
       "      <td>718394219</td>\n",
       "      <td>17.072000</td>\n",
       "      <td>1141.500000</td>\n",
       "      <td>1033.432731</td>\n",
       "      <td>108.067269</td>\n",
       "      <td>153733</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>9.467128</td>\n",
       "      <td>249</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4.390441</td>\n",
       "      <td>2613.377501</td>\n",
       "      <td>74.829600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7183</th>\n",
       "      <td>2013-02-01</td>\n",
       "      <td>Agency_01</td>\n",
       "      <td>SKU_01</td>\n",
       "      <td>431937346</td>\n",
       "      <td>753938444</td>\n",
       "      <td>19.984000</td>\n",
       "      <td>1141.500000</td>\n",
       "      <td>1065.417195</td>\n",
       "      <td>76.082805</td>\n",
       "      <td>153733</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>6.665160</td>\n",
       "      <td>249</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4.585620</td>\n",
       "      <td>2916.978087</td>\n",
       "      <td>90.036700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8928</th>\n",
       "      <td>2013-03-01</td>\n",
       "      <td>Agency_01</td>\n",
       "      <td>SKU_01</td>\n",
       "      <td>509281531</td>\n",
       "      <td>892192092</td>\n",
       "      <td>24.600000</td>\n",
       "      <td>1179.345820</td>\n",
       "      <td>1101.133633</td>\n",
       "      <td>78.212187</td>\n",
       "      <td>153733</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>music_fest</td>\n",
       "      <td>6.631828</td>\n",
       "      <td>249</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4.895628</td>\n",
       "      <td>3215.061952</td>\n",
       "      <td>130.487150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10588</th>\n",
       "      <td>2013-04-01</td>\n",
       "      <td>Agency_01</td>\n",
       "      <td>SKU_01</td>\n",
       "      <td>532390389</td>\n",
       "      <td>838099501</td>\n",
       "      <td>27.532000</td>\n",
       "      <td>1226.687500</td>\n",
       "      <td>1138.283357</td>\n",
       "      <td>88.404143</td>\n",
       "      <td>153733</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>7.206737</td>\n",
       "      <td>249</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4.992553</td>\n",
       "      <td>3515.822697</td>\n",
       "      <td>130.246150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12260</th>\n",
       "      <td>2013-05-01</td>\n",
       "      <td>Agency_01</td>\n",
       "      <td>SKU_01</td>\n",
       "      <td>551755254</td>\n",
       "      <td>864420003</td>\n",
       "      <td>29.396000</td>\n",
       "      <td>1230.331104</td>\n",
       "      <td>1148.969634</td>\n",
       "      <td>81.361470</td>\n",
       "      <td>153733</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>6.612974</td>\n",
       "      <td>249</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>5.168254</td>\n",
       "      <td>3688.107793</td>\n",
       "      <td>159.051550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8403</th>\n",
       "      <td>2017-02-01</td>\n",
       "      <td>Agency_60</td>\n",
       "      <td>SKU_23</td>\n",
       "      <td>530252010</td>\n",
       "      <td>850913048</td>\n",
       "      <td>25.242657</td>\n",
       "      <td>4261.294565</td>\n",
       "      <td>4087.082609</td>\n",
       "      <td>174.211956</td>\n",
       "      <td>2180611</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>4.088240</td>\n",
       "      <td>190</td>\n",
       "      <td>49</td>\n",
       "      <td>2</td>\n",
       "      <td>0.924259</td>\n",
       "      <td>2.418750</td>\n",
       "      <td>2664.670179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10359</th>\n",
       "      <td>2017-03-01</td>\n",
       "      <td>Agency_60</td>\n",
       "      <td>SKU_23</td>\n",
       "      <td>613143990</td>\n",
       "      <td>886129111</td>\n",
       "      <td>25.374816</td>\n",
       "      <td>4259.769000</td>\n",
       "      <td>4126.776000</td>\n",
       "      <td>132.993000</td>\n",
       "      <td>2180611</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>music_fest</td>\n",
       "      <td>3.122071</td>\n",
       "      <td>190</td>\n",
       "      <td>50</td>\n",
       "      <td>3</td>\n",
       "      <td>0.536493</td>\n",
       "      <td>4.353750</td>\n",
       "      <td>2965.472829</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12114</th>\n",
       "      <td>2017-04-01</td>\n",
       "      <td>Agency_60</td>\n",
       "      <td>SKU_23</td>\n",
       "      <td>589969396</td>\n",
       "      <td>940912941</td>\n",
       "      <td>27.109204</td>\n",
       "      <td>4261.896428</td>\n",
       "      <td>4115.753572</td>\n",
       "      <td>146.142856</td>\n",
       "      <td>2180611</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>3.429057</td>\n",
       "      <td>190</td>\n",
       "      <td>51</td>\n",
       "      <td>4</td>\n",
       "      <td>0.231112</td>\n",
       "      <td>2.396250</td>\n",
       "      <td>2861.802300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13884</th>\n",
       "      <td>2017-05-01</td>\n",
       "      <td>Agency_60</td>\n",
       "      <td>SKU_23</td>\n",
       "      <td>628759461</td>\n",
       "      <td>917412482</td>\n",
       "      <td>28.479272</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2180611</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>190</td>\n",
       "      <td>52</td>\n",
       "      <td>5</td>\n",
       "      <td>-18.420681</td>\n",
       "      <td>2.182500</td>\n",
       "      <td>3489.190286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15669</th>\n",
       "      <td>2017-06-01</td>\n",
       "      <td>Agency_60</td>\n",
       "      <td>SKU_23</td>\n",
       "      <td>636846973</td>\n",
       "      <td>928366256</td>\n",
       "      <td>29.609259</td>\n",
       "      <td>4256.675000</td>\n",
       "      <td>4246.018750</td>\n",
       "      <td>10.656250</td>\n",
       "      <td>2180611</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-</td>\n",
       "      <td>-</td>\n",
       "      <td>0.250342</td>\n",
       "      <td>190</td>\n",
       "      <td>53</td>\n",
       "      <td>6</td>\n",
       "      <td>0.924259</td>\n",
       "      <td>2.362500</td>\n",
       "      <td>3423.810793</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>18900 rows × 30 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            date     agency     sku  industry_volume  soda_volume  \\\n",
       "25    2013-01-01  Agency_01  SKU_01        492612703    718394219   \n",
       "7183  2013-02-01  Agency_01  SKU_01        431937346    753938444   \n",
       "8928  2013-03-01  Agency_01  SKU_01        509281531    892192092   \n",
       "10588 2013-04-01  Agency_01  SKU_01        532390389    838099501   \n",
       "12260 2013-05-01  Agency_01  SKU_01        551755254    864420003   \n",
       "...          ...        ...     ...              ...          ...   \n",
       "8403  2017-02-01  Agency_60  SKU_23        530252010    850913048   \n",
       "10359 2017-03-01  Agency_60  SKU_23        613143990    886129111   \n",
       "12114 2017-04-01  Agency_60  SKU_23        589969396    940912941   \n",
       "13884 2017-05-01  Agency_60  SKU_23        628759461    917412482   \n",
       "15669 2017-06-01  Agency_60  SKU_23        636846973    928366256   \n",
       "\n",
       "       avg_max_temp  price_regular  price_actual    discount  \\\n",
       "25        17.072000    1141.500000   1033.432731  108.067269   \n",
       "7183      19.984000    1141.500000   1065.417195   76.082805   \n",
       "8928      24.600000    1179.345820   1101.133633   78.212187   \n",
       "10588     27.532000    1226.687500   1138.283357   88.404143   \n",
       "12260     29.396000    1230.331104   1148.969634   81.361470   \n",
       "...             ...            ...           ...         ...   \n",
       "8403      25.242657    4261.294565   4087.082609  174.211956   \n",
       "10359     25.374816    4259.769000   4126.776000  132.993000   \n",
       "12114     27.109204    4261.896428   4115.753572  146.142856   \n",
       "13884     28.479272       0.000000      0.000000    0.000000   \n",
       "15669     29.609259    4256.675000   4246.018750   10.656250   \n",
       "\n",
       "       avg_population_2017  ...  football_gold_cup beer_capital  music_fest  \\\n",
       "25                  153733  ...                  0            -           -   \n",
       "7183                153733  ...                  0            -           -   \n",
       "8928                153733  ...                  0            -  music_fest   \n",
       "10588               153733  ...                  0            -           -   \n",
       "12260               153733  ...                  0            -           -   \n",
       "...                    ...  ...                ...          ...         ...   \n",
       "8403               2180611  ...                  0            -           -   \n",
       "10359              2180611  ...                  0            -  music_fest   \n",
       "12114              2180611  ...                  0            -           -   \n",
       "13884              2180611  ...                  0            -           -   \n",
       "15669              2180611  ...                  0            -           -   \n",
       "\n",
       "      discount_in_percent timeseries time_idx month log_volume  \\\n",
       "25               9.467128        249        0     1   4.390441   \n",
       "7183             6.665160        249        1     2   4.585620   \n",
       "8928             6.631828        249        2     3   4.895628   \n",
       "10588            7.206737        249        3     4   4.992553   \n",
       "12260            6.612974        249        4     5   5.168254   \n",
       "...                   ...        ...      ...   ...        ...   \n",
       "8403             4.088240        190       49     2   0.924259   \n",
       "10359            3.122071        190       50     3   0.536493   \n",
       "12114            3.429057        190       51     4   0.231112   \n",
       "13884            0.000000        190       52     5 -18.420681   \n",
       "15669            0.250342        190       53     6   0.924259   \n",
       "\n",
       "      avg_volume_by_sku  avg_volume_by_agency  \n",
       "25          2613.377501             74.829600  \n",
       "7183        2916.978087             90.036700  \n",
       "8928        3215.061952            130.487150  \n",
       "10588       3515.822697            130.246150  \n",
       "12260       3688.107793            159.051550  \n",
       "...                 ...                   ...  \n",
       "8403           2.418750           2664.670179  \n",
       "10359          4.353750           2965.472829  \n",
       "12114          2.396250           2861.802300  \n",
       "13884          2.182500           3489.190286  \n",
       "15669          2.362500           3423.810793  \n",
       "\n",
       "[18900 rows x 30 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run FLAML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[flaml.automl: 11-07 02:01:31] {1032} WARNING - Missing timestamps detected. To avoid error with estimators, set estimator list to ['prophet']. \n",
      "[flaml.automl: 11-07 02:01:31] {2600} INFO - task = ts_forecast_panel\n",
      "[flaml.automl: 11-07 02:01:31] {2602} INFO - Data split method: time\n",
      "[flaml.automl: 11-07 02:01:31] {2605} INFO - Evaluation method: holdout\n",
      "[flaml.automl: 11-07 02:01:31] {2727} INFO - Minimizing error metric: mape\n",
      "[flaml.automl: 11-07 02:01:31] {2869} INFO - List of ML learners in AutoML Run: ['tft']\n",
      "[flaml.automl: 11-07 02:01:31] {3164} INFO - iteration 0, current learner tft\n",
      "GPU available: True (cuda), used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "Missing logger folder: lightning_logs/lightning_logs\n",
      "\n",
      "   | Name                               | Type                            | Params\n",
      "----------------------------------------------------------------------------------------\n",
      "0  | loss                               | QuantileLoss                    | 0     \n",
      "1  | logging_metrics                    | ModuleList                      | 0     \n",
      "2  | input_embeddings                   | MultiEmbedding                  | 1.3 K \n",
      "3  | prescalers                         | ModuleDict                      | 256   \n",
      "4  | static_variable_selection          | VariableSelectionNetwork        | 3.4 K \n",
      "5  | encoder_variable_selection         | VariableSelectionNetwork        | 8.0 K \n",
      "6  | decoder_variable_selection         | VariableSelectionNetwork        | 2.7 K \n",
      "7  | static_context_variable_selection  | GatedResidualNetwork            | 1.1 K \n",
      "8  | static_context_initial_hidden_lstm | GatedResidualNetwork            | 1.1 K \n",
      "9  | static_context_initial_cell_lstm   | GatedResidualNetwork            | 1.1 K \n",
      "10 | static_context_enrichment          | GatedResidualNetwork            | 1.1 K \n",
      "11 | lstm_encoder                       | LSTM                            | 4.4 K \n",
      "12 | lstm_decoder                       | LSTM                            | 4.4 K \n",
      "13 | post_lstm_gate_encoder             | GatedLinearUnit                 | 544   \n",
      "14 | post_lstm_add_norm_encoder         | AddNorm                         | 32    \n",
      "15 | static_enrichment                  | GatedResidualNetwork            | 1.4 K \n",
      "16 | multihead_attn                     | InterpretableMultiHeadAttention | 676   \n",
      "17 | post_attn_gate_norm                | GateAddNorm                     | 576   \n",
      "18 | pos_wise_ff                        | GatedResidualNetwork            | 1.1 K \n",
      "19 | pre_output_gate_norm               | GateAddNorm                     | 576   \n",
      "20 | output_layer                       | Linear                          | 119   \n",
      "----------------------------------------------------------------------------------------\n",
      "33.6 K    Trainable params\n",
      "0         Non-trainable params\n",
      "33.6 K    Total params\n",
      "0.135     Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.011338949203491211,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Sanity Checking",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f24513af4d644fe89f9e2a9ba0a7b50c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Sanity Checking: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010900259017944336,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Training",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8182174eee7e4a4b8e15ab302832fab7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010015249252319336,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "06201f86ad9b4d04846f404a7489303c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.00987863540649414,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4dc1b8cd7aaa4ad5af9f197775c323cd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010442733764648438,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "243a4c6b195147b5929e0a0479e9ca07",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.01085972785949707,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4bd55a2c834d40e4a413f4c516b11484",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010792255401611328,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0e4f8ba63bd742cf80106d35a1ef9891",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010687828063964844,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e3921c0faea3440e9a1f117c191a820a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010504722595214844,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c21832c5f022457f8a093159e4dc24e6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.013760089874267578,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fad2fe6e881746ef939deceba61c2bb1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.011274576187133789,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "467953dbea9348668b243574fbc4ca77",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.0159909725189209,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0c29a0db485046eaa2175587ec24640b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010911703109741211,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6f5b0d508e994aa09fd02f7d0749db05",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010562896728515625,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2461c6dd140844b9bd7a10f1e06ea5e3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010767221450805664,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "16ede2b956fe453287c91a9729411391",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.009951353073120117,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e4a3a4bbc2dc4b508564b729c7a64829",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010424613952636719,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7d4b8161b40e4a96b5d73ba7a72db94b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.009989261627197266,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "681b69b02529475c94461ec184d9f554",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010727405548095703,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ce7f2fbb74fe416c90f05e36d4e72a82",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.013596534729003906,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a9ed61fe99594056bfe4a9111e0eb711",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010007858276367188,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3c6103189e1c427487536eae734d08dd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010609626770019531,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e423d2e464a24fc0a26d664732ab26e0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`Trainer.fit` stopped: `max_epochs=20` reached.\n",
      "[flaml.automl: 11-07 02:08:25] {3297} INFO - Estimated sufficient time budget=4131042s. Estimated necessary time budget=4131s.\n",
      "[flaml.automl: 11-07 02:08:25] {3344} INFO -  at 413.2s,\testimator tft's best error=795900256158560.7500,\tbest estimator tft's best error=795900256158560.7500\n",
      "GPU available: True (cuda), used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "\n",
      "   | Name                               | Type                            | Params\n",
      "----------------------------------------------------------------------------------------\n",
      "0  | loss                               | QuantileLoss                    | 0     \n",
      "1  | logging_metrics                    | ModuleList                      | 0     \n",
      "2  | input_embeddings                   | MultiEmbedding                  | 1.3 K \n",
      "3  | prescalers                         | ModuleDict                      | 256   \n",
      "4  | static_variable_selection          | VariableSelectionNetwork        | 3.4 K \n",
      "5  | encoder_variable_selection         | VariableSelectionNetwork        | 8.0 K \n",
      "6  | decoder_variable_selection         | VariableSelectionNetwork        | 2.7 K \n",
      "7  | static_context_variable_selection  | GatedResidualNetwork            | 1.1 K \n",
      "8  | static_context_initial_hidden_lstm | GatedResidualNetwork            | 1.1 K \n",
      "9  | static_context_initial_cell_lstm   | GatedResidualNetwork            | 1.1 K \n",
      "10 | static_context_enrichment          | GatedResidualNetwork            | 1.1 K \n",
      "11 | lstm_encoder                       | LSTM                            | 4.4 K \n",
      "12 | lstm_decoder                       | LSTM                            | 4.4 K \n",
      "13 | post_lstm_gate_encoder             | GatedLinearUnit                 | 544   \n",
      "14 | post_lstm_add_norm_encoder         | AddNorm                         | 32    \n",
      "15 | static_enrichment                  | GatedResidualNetwork            | 1.4 K \n",
      "16 | multihead_attn                     | InterpretableMultiHeadAttention | 676   \n",
      "17 | post_attn_gate_norm                | GateAddNorm                     | 576   \n",
      "18 | pos_wise_ff                        | GatedResidualNetwork            | 1.1 K \n",
      "19 | pre_output_gate_norm               | GateAddNorm                     | 576   \n",
      "20 | output_layer                       | Linear                          | 119   \n",
      "----------------------------------------------------------------------------------------\n",
      "33.6 K    Trainable params\n",
      "0         Non-trainable params\n",
      "33.6 K    Total params\n",
      "0.135     Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.01064157485961914,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Sanity Checking",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "039bb3197d7644959046cbe4e606d661",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Sanity Checking: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010631084442138672,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Training",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f32a1341cfae4812a0862f9ca0071071",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010550498962402344,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "69262e0dbd944892a82b7601eca38b4b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.01083064079284668,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ffd76edb2480405a840d84602f97565c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.013179302215576172,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "11c20f5b8a5147809d2a1bf703b2ff77",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010700225830078125,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b27e8008a03d4570b4a6f894143127ab",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010641813278198242,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d8aa1ace332949ed9d26024c8346983f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.009891510009765625,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "04c4f8969da248e8a70ca3dd16d40a29",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.013002157211303711,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6d03ebc520fe4a958082e2fad30e1456",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.013621091842651367,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b41649defc12434ba4ccb1be452ac0b1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010574102401733398,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3820a4248f0a49f38c4978649b89535c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010617971420288086,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dc0d16e9a517444ca7092a9a705fdcd8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.011349916458129883,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "38035d69d18042aa9a568cebbb803218",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.009980201721191406,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f3dc9ca0f6984696ba0d42a5fdd8fb6e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.011469602584838867,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "52bc87f1224746c0a59a347ab5a2569d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.012477397918701172,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "99eb7a20d6584575854b434e2813063b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.01574850082397461,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5b81bee41485454285ce00a587486289",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.01217198371887207,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "eb2468f1f1c44b7188f19b422ff24a9a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.010460138320922852,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "46fd834aa8894298b4a2af5d20824236",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/json": {
       "ascii": false,
       "bar_format": null,
       "colour": null,
       "elapsed": 0.00981903076171875,
       "initial": 0,
       "n": 0,
       "ncols": null,
       "nrows": null,
       "postfix": null,
       "prefix": "Validation",
       "rate": null,
       "total": null,
       "unit": "it",
       "unit_divisor": 1000,
       "unit_scale": false
      },
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "87bb91504c0e4ea889c711cd8d35a08a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[flaml.automl: 11-07 02:15:24] {3608} INFO - retrain tft for 419.5s\n",
      "[flaml.automl: 11-07 02:15:24] {3615} INFO - retrained model: TemporalFusionTransformer(\n",
      "  \t\"attention_head_size\":               4\n",
      "  \t\"categorical_groups\":                {'special_days': ['easter_day', 'good_friday', 'new_year', 'christmas', 'labor_day', 'independence_day', 'revolution_day_memorial', 'regional_games', 'beer_capital', 'music_fest']}\n",
      "  \t\"causal_attention\":                  True\n",
      "  \t\"dropout\":                           0.1\n",
      "  \t\"embedding_labels\":                  {'agency': {'Agency_01': 0, 'Agency_02': 1, 'Agency_03': 2, 'Agency_04': 3, 'Agency_05': 4, 'Agency_07': 5, 'Agency_08': 6, 'Agency_09': 7, 'Agency_10': 8, 'Agency_11': 9, 'Agency_12': 10, 'Agency_13': 11, 'Agency_15': 12, 'Agency_16': 13, 'Agency_17': 14, 'Agency_18': 15, 'Agency_19': 16, 'Agency_20': 17, 'Agency_21': 18, 'Agency_22': 19, 'Agency_23': 20, 'Agency_24': 21, 'Agency_25': 22, 'Agency_26': 23, 'Agency_27': 24, 'Agency_28': 25, 'Agency_29': 26, 'Agency_30': 27, 'Agency_31': 28, 'Agency_32': 29, 'Agency_33': 30, 'Agency_34': 31, 'Agency_35': 32, 'Agency_36': 33, 'Agency_37': 34, 'Agency_38': 35, 'Agency_39': 36, 'Agency_40': 37, 'Agency_41': 38, 'Agency_42': 39, 'Agency_43': 40, 'Agency_44': 41, 'Agency_45': 42, 'Agency_46': 43, 'Agency_47': 44, 'Agency_48': 45, 'Agency_49': 46, 'Agency_50': 47, 'Agency_51': 48, 'Agency_52': 49, 'Agency_53': 50, 'Agency_54': 51, 'Agency_55': 52, 'Agency_56': 53, 'Agency_57': 54, 'Agency_58': 55, 'Agency_59': 56, 'Agency_60': 57}, 'sku': {'SKU_01': 0, 'SKU_02': 1, 'SKU_03': 2, 'SKU_04': 3, 'SKU_05': 4, 'SKU_06': 5, 'SKU_07': 6, 'SKU_08': 7, 'SKU_11': 8, 'SKU_12': 9, 'SKU_14': 10, 'SKU_15': 11, 'SKU_17': 12, 'SKU_18': 13, 'SKU_20': 14, 'SKU_21': 15, 'SKU_22': 16, 'SKU_23': 17, 'SKU_24': 18, 'SKU_26': 19, 'SKU_27': 20, 'SKU_28': 21, 'SKU_31': 22, 'SKU_32': 23, 'SKU_34': 24}, 'special_days': {'-': 0, 'beer_capital': 1, 'christmas': 2, 'easter_day': 3, 'good_friday': 4, 'independence_day': 5, 'labor_day': 6, 'music_fest': 7, 'new_year': 8, 'regional_games': 9, 'revolution_day_memorial': 10}, 'month': {'1': 0, '10': 1, '11': 2, '12': 3, '2': 4, '3': 5, '4': 6, '5': 7, '6': 8, '7': 9, '8': 10, '9': 11}}\n",
      "  \t\"embedding_paddings\":                []\n",
      "  \t\"embedding_sizes\":                   {'agency': (58, 16), 'sku': (25, 10), 'special_days': (11, 6), 'month': (12, 6)}\n",
      "  \t\"hidden_continuous_size\":            8\n",
      "  \t\"hidden_continuous_sizes\":           {}\n",
      "  \t\"hidden_size\":                       16\n",
      "  \t\"learning_rate\":                     0.0010000000000000002\n",
      "  \t\"log_gradient_flow\":                 False\n",
      "  \t\"log_interval\":                      10\n",
      "  \t\"log_val_interval\":                  10\n",
      "  \t\"logging_metrics\":                   ModuleList(\n",
      "  \t  (0): SMAPE()\n",
      "  \t  (1): MAE()\n",
      "  \t  (2): RMSE()\n",
      "  \t  (3): MAPE()\n",
      "  \t)\n",
      "  \t\"loss\":                              QuantileLoss(quantiles=[0.02, 0.1, 0.25, 0.5, 0.75, 0.9, 0.98])\n",
      "  \t\"lstm_layers\":                       2\n",
      "  \t\"max_encoder_length\":                24\n",
      "  \t\"monotone_constaints\":               {}\n",
      "  \t\"optimizer\":                         ranger\n",
      "  \t\"optimizer_params\":                  None\n",
      "  \t\"output_size\":                       7\n",
      "  \t\"output_transformer\":                GroupNormalizer(\n",
      "  \t\tmethod='standard',\n",
      "  \t\tgroups=['agency', 'sku'],\n",
      "  \t\tcenter=True,\n",
      "  \t\tscale_by_group=False,\n",
      "  \t\ttransformation='softplus',\n",
      "  \t\tmethod_kwargs={}\n",
      "  \t)\n",
      "  \t\"reduce_on_plateau_min_lr\":          1e-05\n",
      "  \t\"reduce_on_plateau_patience\":        4\n",
      "  \t\"reduce_on_plateau_reduction\":       2.0\n",
      "  \t\"share_single_variable_networks\":    False\n",
      "  \t\"static_categoricals\":               ['agency', 'sku']\n",
      "  \t\"static_reals\":                      ['avg_population_2017', 'avg_yearly_household_income_2017', 'encoder_length', 'y_center', 'y_scale']\n",
      "  \t\"time_varying_categoricals_decoder\": ['special_days', 'month']\n",
      "  \t\"time_varying_categoricals_encoder\": ['special_days', 'month']\n",
      "  \t\"time_varying_reals_decoder\":        ['time_idx', 'price_regular', 'discount_in_percent', 'relative_time_idx']\n",
      "  \t\"time_varying_reals_encoder\":        ['time_idx', 'price_regular', 'discount_in_percent', 'relative_time_idx', 'y', 'log_volume', 'industry_volume', 'soda_volume', 'avg_max_temp', 'avg_volume_by_agency', 'avg_volume_by_sku']\n",
      "  \t\"weight_decay\":                      0.0\n",
      "  \t\"x_categoricals\":                    ['agency', 'sku', 'easter_day', 'good_friday', 'new_year', 'christmas', 'labor_day', 'independence_day', 'revolution_day_memorial', 'regional_games', 'beer_capital', 'music_fest', 'month']\n",
      "  \t\"x_reals\":                           ['avg_population_2017', 'avg_yearly_household_income_2017', 'encoder_length', 'y_center', 'y_scale', 'time_idx', 'price_regular', 'discount_in_percent', 'relative_time_idx', 'y', 'log_volume', 'industry_volume', 'soda_volume', 'avg_max_temp', 'avg_volume_by_agency', 'avg_volume_by_sku']\n",
      "  (loss): QuantileLoss(quantiles=[0.02, 0.1, 0.25, 0.5, 0.75, 0.9, 0.98])\n",
      "  (logging_metrics): ModuleList(\n",
      "    (0): SMAPE()\n",
      "    (1): MAE()\n",
      "    (2): RMSE()\n",
      "    (3): MAPE()\n",
      "  )\n",
      "  (input_embeddings): MultiEmbedding(\n",
      "    (embeddings): ModuleDict(\n",
      "      (agency): Embedding(58, 16)\n",
      "      (sku): Embedding(25, 10)\n",
      "      (special_days): TimeDistributedEmbeddingBag(11, 6, mode=sum)\n",
      "      (month): Embedding(12, 6)\n",
      "    )\n",
      "  )\n",
      "  (prescalers): ModuleDict(\n",
      "    (avg_population_2017): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (avg_yearly_household_income_2017): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (encoder_length): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (y_center): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (y_scale): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (time_idx): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (price_regular): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (discount_in_percent): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (relative_time_idx): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (y): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (log_volume): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (industry_volume): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (soda_volume): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (avg_max_temp): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (avg_volume_by_agency): Linear(in_features=1, out_features=8, bias=True)\n",
      "    (avg_volume_by_sku): Linear(in_features=1, out_features=8, bias=True)\n",
      "  )\n",
      "  (static_variable_selection): VariableSelectionNetwork(\n",
      "    (flattened_grn): GatedResidualNetwork(\n",
      "      (resample_norm): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((7,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (fc1): Linear(in_features=66, out_features=7, bias=True)\n",
      "      (elu): ELU(alpha=1.0)\n",
      "      (fc2): Linear(in_features=7, out_features=7, bias=True)\n",
      "      (gate_norm): GateAddNorm(\n",
      "        (glu): GatedLinearUnit(\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "          (fc): Linear(in_features=7, out_features=14, bias=True)\n",
      "        )\n",
      "        (add_norm): AddNorm(\n",
      "          (norm): LayerNorm((7,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (single_variable_grns): ModuleDict(\n",
      "      (agency): ResampleNorm(\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (sku): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (avg_population_2017): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (avg_yearly_household_income_2017): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (encoder_length): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (y_center): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (y_scale): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (prescalers): ModuleDict(\n",
      "      (avg_population_2017): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (avg_yearly_household_income_2017): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (encoder_length): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (y_center): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (y_scale): Linear(in_features=1, out_features=8, bias=True)\n",
      "    )\n",
      "    (softmax): Softmax(dim=-1)\n",
      "  )\n",
      "  (encoder_variable_selection): VariableSelectionNetwork(\n",
      "    (flattened_grn): GatedResidualNetwork(\n",
      "      (resample_norm): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((13,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (fc1): Linear(in_features=100, out_features=13, bias=True)\n",
      "      (elu): ELU(alpha=1.0)\n",
      "      (context): Linear(in_features=16, out_features=13, bias=False)\n",
      "      (fc2): Linear(in_features=13, out_features=13, bias=True)\n",
      "      (gate_norm): GateAddNorm(\n",
      "        (glu): GatedLinearUnit(\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "          (fc): Linear(in_features=13, out_features=26, bias=True)\n",
      "        )\n",
      "        (add_norm): AddNorm(\n",
      "          (norm): LayerNorm((13,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (single_variable_grns): ModuleDict(\n",
      "      (special_days): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (month): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (time_idx): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (price_regular): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (discount_in_percent): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (relative_time_idx): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (y): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (log_volume): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (industry_volume): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (soda_volume): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (avg_max_temp): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (avg_volume_by_agency): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (avg_volume_by_sku): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (prescalers): ModuleDict(\n",
      "      (time_idx): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (price_regular): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (discount_in_percent): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (relative_time_idx): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (y): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (log_volume): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (industry_volume): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (soda_volume): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (avg_max_temp): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (avg_volume_by_agency): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (avg_volume_by_sku): Linear(in_features=1, out_features=8, bias=True)\n",
      "    )\n",
      "    (softmax): Softmax(dim=-1)\n",
      "  )\n",
      "  (decoder_variable_selection): VariableSelectionNetwork(\n",
      "    (flattened_grn): GatedResidualNetwork(\n",
      "      (resample_norm): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((6,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (fc1): Linear(in_features=44, out_features=6, bias=True)\n",
      "      (elu): ELU(alpha=1.0)\n",
      "      (context): Linear(in_features=16, out_features=6, bias=False)\n",
      "      (fc2): Linear(in_features=6, out_features=6, bias=True)\n",
      "      (gate_norm): GateAddNorm(\n",
      "        (glu): GatedLinearUnit(\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "          (fc): Linear(in_features=6, out_features=12, bias=True)\n",
      "        )\n",
      "        (add_norm): AddNorm(\n",
      "          (norm): LayerNorm((6,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (single_variable_grns): ModuleDict(\n",
      "      (special_days): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (month): ResampleNorm(\n",
      "        (resample): TimeDistributedInterpolation()\n",
      "        (gate): Sigmoid()\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "      (time_idx): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (price_regular): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (discount_in_percent): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "      (relative_time_idx): GatedResidualNetwork(\n",
      "        (resample_norm): ResampleNorm(\n",
      "          (resample): TimeDistributedInterpolation()\n",
      "          (gate): Sigmoid()\n",
      "          (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "        )\n",
      "        (fc1): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (elu): ELU(alpha=1.0)\n",
      "        (fc2): Linear(in_features=8, out_features=8, bias=True)\n",
      "        (gate_norm): GateAddNorm(\n",
      "          (glu): GatedLinearUnit(\n",
      "            (dropout): Dropout(p=0.1, inplace=False)\n",
      "            (fc): Linear(in_features=8, out_features=32, bias=True)\n",
      "          )\n",
      "          (add_norm): AddNorm(\n",
      "            (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "          )\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (prescalers): ModuleDict(\n",
      "      (time_idx): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (price_regular): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (discount_in_percent): Linear(in_features=1, out_features=8, bias=True)\n",
      "      (relative_time_idx): Linear(in_features=1, out_features=8, bias=True)\n",
      "    )\n",
      "    (softmax): Softmax(dim=-1)\n",
      "  )\n",
      "  (static_context_variable_selection): GatedResidualNetwork(\n",
      "    (fc1): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (elu): ELU(alpha=1.0)\n",
      "    (fc2): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (gate_norm): GateAddNorm(\n",
      "      (glu): GatedLinearUnit(\n",
      "        (dropout): Dropout(p=0.1, inplace=False)\n",
      "        (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "      )\n",
      "      (add_norm): AddNorm(\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "    )\n",
      "  )\n",
      "  (static_context_initial_hidden_lstm): GatedResidualNetwork(\n",
      "    (fc1): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (elu): ELU(alpha=1.0)\n",
      "    (fc2): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (gate_norm): GateAddNorm(\n",
      "      (glu): GatedLinearUnit(\n",
      "        (dropout): Dropout(p=0.1, inplace=False)\n",
      "        (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "      )\n",
      "      (add_norm): AddNorm(\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "    )\n",
      "  )\n",
      "  (static_context_initial_cell_lstm): GatedResidualNetwork(\n",
      "    (fc1): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (elu): ELU(alpha=1.0)\n",
      "    (fc2): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (gate_norm): GateAddNorm(\n",
      "      (glu): GatedLinearUnit(\n",
      "        (dropout): Dropout(p=0.1, inplace=False)\n",
      "        (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "      )\n",
      "      (add_norm): AddNorm(\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "    )\n",
      "  )\n",
      "  (static_context_enrichment): GatedResidualNetwork(\n",
      "    (fc1): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (elu): ELU(alpha=1.0)\n",
      "    (fc2): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (gate_norm): GateAddNorm(\n",
      "      (glu): GatedLinearUnit(\n",
      "        (dropout): Dropout(p=0.1, inplace=False)\n",
      "        (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "      )\n",
      "      (add_norm): AddNorm(\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "    )\n",
      "  )\n",
      "  (lstm_encoder): LSTM(16, 16, num_layers=2, batch_first=True, dropout=0.1)\n",
      "  (lstm_decoder): LSTM(16, 16, num_layers=2, batch_first=True, dropout=0.1)\n",
      "  (post_lstm_gate_encoder): GatedLinearUnit(\n",
      "    (dropout): Dropout(p=0.1, inplace=False)\n",
      "    (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "  )\n",
      "  (post_lstm_gate_decoder): GatedLinearUnit(\n",
      "    (dropout): Dropout(p=0.1, inplace=False)\n",
      "    (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "  )\n",
      "  (post_lstm_add_norm_encoder): AddNorm(\n",
      "    (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "  )\n",
      "  (post_lstm_add_norm_decoder): AddNorm(\n",
      "    (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "  )\n",
      "  (static_enrichment): GatedResidualNetwork(\n",
      "    (fc1): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (elu): ELU(alpha=1.0)\n",
      "    (context): Linear(in_features=16, out_features=16, bias=False)\n",
      "    (fc2): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (gate_norm): GateAddNorm(\n",
      "      (glu): GatedLinearUnit(\n",
      "        (dropout): Dropout(p=0.1, inplace=False)\n",
      "        (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "      )\n",
      "      (add_norm): AddNorm(\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "    )\n",
      "  )\n",
      "  (multihead_attn): InterpretableMultiHeadAttention(\n",
      "    (dropout): Dropout(p=0.1, inplace=False)\n",
      "    (v_layer): Linear(in_features=16, out_features=4, bias=True)\n",
      "    (q_layers): ModuleList(\n",
      "      (0): Linear(in_features=16, out_features=4, bias=True)\n",
      "      (1): Linear(in_features=16, out_features=4, bias=True)\n",
      "      (2): Linear(in_features=16, out_features=4, bias=True)\n",
      "      (3): Linear(in_features=16, out_features=4, bias=True)\n",
      "    )\n",
      "    (k_layers): ModuleList(\n",
      "      (0): Linear(in_features=16, out_features=4, bias=True)\n",
      "      (1): Linear(in_features=16, out_features=4, bias=True)\n",
      "      (2): Linear(in_features=16, out_features=4, bias=True)\n",
      "      (3): Linear(in_features=16, out_features=4, bias=True)\n",
      "    )\n",
      "    (attention): ScaledDotProductAttention(\n",
      "      (softmax): Softmax(dim=2)\n",
      "    )\n",
      "    (w_h): Linear(in_features=4, out_features=16, bias=False)\n",
      "  )\n",
      "  (post_attn_gate_norm): GateAddNorm(\n",
      "    (glu): GatedLinearUnit(\n",
      "      (dropout): Dropout(p=0.1, inplace=False)\n",
      "      (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "    )\n",
      "    (add_norm): AddNorm(\n",
      "      (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "    )\n",
      "  )\n",
      "  (pos_wise_ff): GatedResidualNetwork(\n",
      "    (fc1): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (elu): ELU(alpha=1.0)\n",
      "    (fc2): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (gate_norm): GateAddNorm(\n",
      "      (glu): GatedLinearUnit(\n",
      "        (dropout): Dropout(p=0.1, inplace=False)\n",
      "        (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "      )\n",
      "      (add_norm): AddNorm(\n",
      "        (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "      )\n",
      "    )\n",
      "  )\n",
      "  (pre_output_gate_norm): GateAddNorm(\n",
      "    (glu): GatedLinearUnit(\n",
      "      (fc): Linear(in_features=16, out_features=32, bias=True)\n",
      "    )\n",
      "    (add_norm): AddNorm(\n",
      "      (norm): LayerNorm((16,), eps=1e-05, elementwise_affine=True)\n",
      "    )\n",
      "  )\n",
      "  (output_layer): Linear(in_features=16, out_features=7, bias=True)\n",
      ")\n",
      "[flaml.automl: 11-07 02:15:24] {2900} INFO - fit succeeded\n",
      "[flaml.automl: 11-07 02:15:24] {2901} INFO - Time taken to find the best model: 413.17405128479004\n",
      "[flaml.automl: 11-07 02:15:24] {2912} WARNING - Time taken to find the best model is 138% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.\n"
     ]
    }
   ],
   "source": [
    "from flaml import AutoML\n",
    "automl = AutoML()\n",
    "settings = {\n",
    "    \"time_budget\": 300,  # total running time in seconds\n",
    "    \"metric\": \"mape\",  # primary metric\n",
    "    \"task\": \"ts_forecast_panel\",  # task type\n",
    "    \"log_file_name\": \"stallion_forecast.log\",  # flaml log file\n",
    "    \"eval_method\": \"holdout\",\n",
    "}\n",
    "fit_kwargs_by_estimator = {\n",
    "    \"tft\": {\n",
    "        \"max_encoder_length\": 24,\n",
    "        \"static_categoricals\": [\"agency\", \"sku\"],\n",
    "        \"static_reals\": [\"avg_population_2017\", \"avg_yearly_household_income_2017\"],\n",
    "        \"time_varying_known_categoricals\": [\"special_days\", \"month\"],\n",
    "        \"variable_groups\": {\n",
    "            \"special_days\": special_days\n",
    "        },  # group of categorical variables can be treated as one variable\n",
    "        \"time_varying_known_reals\": [\n",
    "            \"time_idx\",\n",
    "            \"price_regular\",\n",
    "            \"discount_in_percent\",\n",
    "        ],\n",
    "        \"time_varying_unknown_categoricals\": [],\n",
    "        \"time_varying_unknown_reals\": [\n",
    "            \"y\",  # always need a 'y' column for the target column\n",
    "            \"log_volume\",\n",
    "            \"industry_volume\",\n",
    "            \"soda_volume\",\n",
    "            \"avg_max_temp\",\n",
    "            \"avg_volume_by_agency\",\n",
    "            \"avg_volume_by_sku\",\n",
    "        ],\n",
    "        \"batch_size\": 128,\n",
    "        \"gpu_per_trial\": 0,\n",
    "    }\n",
    "}\n",
    "\"\"\"The main flaml automl API\"\"\"\n",
    "automl.fit(\n",
    "    X_train=X_train,\n",
    "    y_train=y_train,\n",
    "    **settings,\n",
    "    period=time_horizon,\n",
    "    group_ids=[\"agency\", \"sku\"],\n",
    "    fit_kwargs_by_estimator=fit_kwargs_by_estimator,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction and Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17156    59.292\n",
      "18946    66.420\n",
      "20680    95.904\n",
      "3189     52.812\n",
      "4954     37.908\n",
      "          ...  \n",
      "19207     1.980\n",
      "20996     1.260\n",
      "3499      0.990\n",
      "5248      0.090\n",
      "6793      2.250\n",
      "Name: volume, Length: 2100, dtype: float64\n",
      "Agency_01  SKU_01  2017-07-01    5.836853e+01\n",
      "                   2017-08-01    5.648019e+01\n",
      "                   2017-09-01    6.513703e+01\n",
      "                   2017-10-01    5.674841e+01\n",
      "                   2017-11-01    4.554249e+01\n",
      "                                     ...     \n",
      "Agency_60  SKU_23  2017-08-01    1.689411e-15\n",
      "                   2017-09-01    1.250672e-10\n",
      "                   2017-10-01    3.494929e-21\n",
      "                   2017-11-01    1.006966e-16\n",
      "                   2017-12-01    1.217613e-21\n",
      "Length: 2100, dtype: float32\n"
     ]
    }
   ],
   "source": [
    "\"\"\" compute predictions of testing dataset \"\"\"\n",
    "y_pred = automl.predict(X_test)\n",
    "print(y_test)\n",
    "print(y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mape = 2718002246141115.0\n",
      "smape = 61.82\n"
     ]
    }
   ],
   "source": [
    "\"\"\" compute different metric values on testing dataset\"\"\"\n",
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print(\"mape\", \"=\", sklearn_metric_loss_score(\"mape\", y_pred, y_test))\n",
    "\n",
    "def smape(y_pred, y_test):\n",
    "    import numpy as np\n",
    "\n",
    "    y_test, y_pred = np.array(y_test), np.array(y_pred)\n",
    "    return round(\n",
    "        np.mean(\n",
    "            np.abs(y_pred - y_test) /\n",
    "            ((np.abs(y_pred) + np.abs(y_test)) / 2)\n",
    "        ) * 100, 2\n",
    "    )\n",
    "\n",
    "print(\"smape\", \"=\", smape(y_pred, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Comparison with Alternatives (CO2 Dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "FLAML's MAPE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "flaml mape = inf\n"
     ]
    }
   ],
   "source": [
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print('flaml mape', '=', sklearn_metric_loss_score('mape', flaml_y_pred, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Default Prophet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "from prophet import Prophet\n",
    "prophet_model = Prophet()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "02:15:59 - cmdstanpy - INFO - Chain [1] start processing\n",
      "02:15:59 - cmdstanpy - INFO - Chain [1] done processing\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<prophet.forecaster.Prophet at 0x7f73c990bc10>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_prophet = train_df.copy()\n",
    "X_train_prophet = X_train_prophet.rename(columns={'index': 'ds', 'co2': 'y'})\n",
    "prophet_model.fit(X_train_prophet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted labels 0     370.451280\n",
      "1     371.177888\n",
      "2     372.230018\n",
      "3     373.420156\n",
      "4     373.914729\n",
      "5     373.406175\n",
      "6     372.054228\n",
      "7     370.149927\n",
      "8     368.567756\n",
      "9     368.647528\n",
      "10    369.864590\n",
      "11    371.137314\n",
      "Name: yhat, dtype: float64\n",
      "True labels 514    370.175\n",
      "515    371.325\n",
      "516    372.060\n",
      "517    372.775\n",
      "518    373.800\n",
      "519    373.060\n",
      "520    371.300\n",
      "521    369.425\n",
      "522    367.880\n",
      "523    368.050\n",
      "524    369.375\n",
      "525    371.020\n",
      "Name: co2, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "X_test_prophet = X_test.copy()\n",
    "X_test_prophet = X_test_prophet.rename(columns={'index': 'ds'})\n",
    "prophet_y_pred = prophet_model.predict(X_test_prophet)['yhat']\n",
    "print('Predicted labels', prophet_y_pred)\n",
    "print('True labels', y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Default Prophet MAPE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "default prophet mape = 0.0011411103714832386\n"
     ]
    }
   ],
   "source": [
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print('default prophet mape', '=', sklearn_metric_loss_score('mape', prophet_y_pred, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Auto ARIMA Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pmdarima.arima import auto_arima\n",
    "import pandas as pd\n",
    "import time\n",
    "\n",
    "X_train_arima = train_df.copy()\n",
    "X_train_arima.index = pd.to_datetime(X_train_arima['index'])\n",
    "X_train_arima = X_train_arima.drop('index', axis=1)\n",
    "X_train_arima = X_train_arima.rename(columns={'co2': 'y'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ARIMA(0,1,0)(0,0,0)[0] intercept   : AIC=1638.009, Time=0.03 sec\n",
      " ARIMA(0,1,1)(0,0,0)[0] intercept   : AIC=1344.207, Time=0.10 sec\n",
      " ARIMA(0,1,2)(0,0,0)[0] intercept   : AIC=1222.286, Time=0.08 sec\n",
      " ARIMA(0,1,3)(0,0,0)[0] intercept   : AIC=1174.928, Time=0.10 sec\n",
      " ARIMA(0,1,4)(0,0,0)[0] intercept   : AIC=1188.947, Time=0.18 sec\n",
      " ARIMA(0,1,5)(0,0,0)[0] intercept   : AIC=1091.452, Time=0.25 sec\n",
      " ARIMA(1,1,0)(0,0,0)[0] intercept   : AIC=1298.693, Time=0.05 sec\n",
      " ARIMA(1,1,1)(0,0,0)[0] intercept   : AIC=1240.963, Time=0.07 sec\n",
      " ARIMA(1,1,2)(0,0,0)[0] intercept   : AIC=1196.535, Time=0.09 sec\n",
      " ARIMA(1,1,3)(0,0,0)[0] intercept   : AIC=1176.484, Time=0.15 sec\n",
      " ARIMA(1,1,4)(0,0,0)[0] intercept   : AIC=inf, Time=0.53 sec\n",
      " ARIMA(2,1,0)(0,0,0)[0] intercept   : AIC=1180.404, Time=0.06 sec\n",
      " ARIMA(2,1,1)(0,0,0)[0] intercept   : AIC=990.719, Time=0.14 sec\n",
      " ARIMA(2,1,2)(0,0,0)[0] intercept   : AIC=988.094, Time=0.31 sec\n",
      " ARIMA(2,1,3)(0,0,0)[0] intercept   : AIC=1140.469, Time=0.25 sec\n",
      " ARIMA(3,1,0)(0,0,0)[0] intercept   : AIC=1126.139, Time=0.11 sec\n",
      " ARIMA(3,1,1)(0,0,0)[0] intercept   : AIC=989.496, Time=0.24 sec\n",
      " ARIMA(3,1,2)(0,0,0)[0] intercept   : AIC=991.558, Time=0.42 sec\n",
      " ARIMA(4,1,0)(0,0,0)[0] intercept   : AIC=1125.025, Time=0.09 sec\n",
      " ARIMA(4,1,1)(0,0,0)[0] intercept   : AIC=988.660, Time=0.42 sec\n",
      " ARIMA(5,1,0)(0,0,0)[0] intercept   : AIC=1113.673, Time=0.10 sec\n",
      "\n",
      "Best model:  ARIMA(2,1,2)(0,0,0)[0] intercept\n",
      "Total fit time: 3.776 seconds\n"
     ]
    }
   ],
   "source": [
    "# use same search space as FLAML\n",
    "start_time = time.time()\n",
    "arima_model = auto_arima(X_train_arima,\n",
    "                            start_p=2, d=None, start_q=1, max_p=10, max_d=10, max_q=10,\n",
    "                            suppress_warnings=True, stepwise=False, seasonal=False,\n",
    "                            error_action='ignore', trace=True, n_fits=650)\n",
    "autoarima_y_pred = arima_model.predict(n_periods=12)\n",
    "arima_time = time.time() - start_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ARIMA(0,1,0)(0,0,0)[12] intercept   : AIC=1638.009, Time=0.04 sec\n",
      " ARIMA(0,1,0)(0,0,1)[12] intercept   : AIC=1238.943, Time=0.17 sec\n",
      " ARIMA(0,1,0)(0,0,2)[12] intercept   : AIC=1040.890, Time=0.38 sec\n",
      " ARIMA(0,1,0)(0,0,3)[12] intercept   : AIC=911.545, Time=1.07 sec\n",
      " ARIMA(0,1,0)(0,0,4)[12] intercept   : AIC=823.103, Time=2.15 sec\n",
      " ARIMA(0,1,0)(0,0,5)[12] intercept   : AIC=792.850, Time=6.01 sec\n",
      " ARIMA(0,1,0)(1,0,0)[12] intercept   : AIC=inf, Time=0.15 sec\n",
      " ARIMA(0,1,0)(1,0,1)[12] intercept   : AIC=inf, Time=0.61 sec\n",
      " ARIMA(0,1,0)(1,0,2)[12] intercept   : AIC=inf, Time=1.55 sec\n",
      " ARIMA(0,1,0)(1,0,3)[12] intercept   : AIC=438.686, Time=3.78 sec\n",
      " ARIMA(0,1,0)(1,0,4)[12] intercept   : AIC=inf, Time=7.15 sec\n",
      " ARIMA(0,1,0)(2,0,0)[12] intercept   : AIC=inf, Time=0.67 sec\n",
      " ARIMA(0,1,0)(2,0,1)[12] intercept   : AIC=inf, Time=1.55 sec\n",
      " ARIMA(0,1,0)(2,0,2)[12] intercept   : AIC=inf, Time=1.79 sec\n",
      " ARIMA(0,1,0)(2,0,3)[12] intercept   : AIC=inf, Time=5.03 sec\n",
      " ARIMA(0,1,0)(3,0,0)[12] intercept   : AIC=inf, Time=2.24 sec\n",
      " ARIMA(0,1,0)(3,0,1)[12] intercept   : AIC=429.059, Time=4.18 sec\n",
      " ARIMA(0,1,0)(3,0,2)[12] intercept   : AIC=431.443, Time=4.51 sec\n",
      " ARIMA(0,1,0)(4,0,0)[12] intercept   : AIC=inf, Time=5.44 sec\n",
      " ARIMA(0,1,0)(4,0,1)[12] intercept   : AIC=430.330, Time=7.88 sec\n",
      " ARIMA(0,1,0)(5,0,0)[12] intercept   : AIC=inf, Time=15.17 sec\n",
      " ARIMA(0,1,1)(0,0,0)[12] intercept   : AIC=1344.207, Time=0.06 sec\n",
      " ARIMA(0,1,1)(0,0,1)[12] intercept   : AIC=1112.274, Time=0.30 sec\n",
      " ARIMA(0,1,1)(0,0,2)[12] intercept   : AIC=993.565, Time=0.57 sec\n",
      " ARIMA(0,1,1)(0,0,3)[12] intercept   : AIC=891.683, Time=1.87 sec\n",
      " ARIMA(0,1,1)(0,0,4)[12] intercept   : AIC=820.025, Time=3.91 sec\n",
      " ARIMA(0,1,1)(1,0,0)[12] intercept   : AIC=612.811, Time=0.31 sec\n",
      " ARIMA(0,1,1)(1,0,1)[12] intercept   : AIC=394.722, Time=0.83 sec\n",
      " ARIMA(0,1,1)(1,0,2)[12] intercept   : AIC=396.738, Time=2.47 sec\n",
      " ARIMA(0,1,1)(1,0,3)[12] intercept   : AIC=421.007, Time=5.62 sec\n",
      " ARIMA(0,1,1)(2,0,0)[12] intercept   : AIC=510.637, Time=1.00 sec\n",
      " ARIMA(0,1,1)(2,0,1)[12] intercept   : AIC=406.663, Time=1.93 sec\n",
      " ARIMA(0,1,1)(2,0,2)[12] intercept   : AIC=396.801, Time=2.54 sec\n",
      " ARIMA(0,1,1)(3,0,0)[12] intercept   : AIC=467.985, Time=3.21 sec\n",
      " ARIMA(0,1,1)(3,0,1)[12] intercept   : AIC=412.750, Time=5.26 sec\n",
      " ARIMA(0,1,1)(4,0,0)[12] intercept   : AIC=448.948, Time=5.02 sec\n",
      " ARIMA(0,1,2)(0,0,0)[12] intercept   : AIC=1222.286, Time=0.09 sec\n",
      " ARIMA(0,1,2)(0,0,1)[12] intercept   : AIC=1046.922, Time=0.24 sec\n",
      " ARIMA(0,1,2)(0,0,2)[12] intercept   : AIC=947.532, Time=0.62 sec\n",
      " ARIMA(0,1,2)(0,0,3)[12] intercept   : AIC=867.310, Time=1.64 sec\n",
      " ARIMA(0,1,2)(1,0,0)[12] intercept   : AIC=608.450, Time=0.41 sec\n",
      " ARIMA(0,1,2)(1,0,1)[12] intercept   : AIC=386.828, Time=0.94 sec\n",
      " ARIMA(0,1,2)(1,0,2)[12] intercept   : AIC=421.311, Time=2.48 sec\n",
      " ARIMA(0,1,2)(2,0,0)[12] intercept   : AIC=507.685, Time=1.23 sec\n",
      " ARIMA(0,1,2)(2,0,1)[12] intercept   : AIC=408.508, Time=2.14 sec\n",
      " ARIMA(0,1,2)(3,0,0)[12] intercept   : AIC=460.596, Time=3.97 sec\n",
      " ARIMA(0,1,3)(0,0,0)[12] intercept   : AIC=1174.928, Time=0.11 sec\n",
      " ARIMA(0,1,3)(0,0,1)[12] intercept   : AIC=1037.324, Time=0.34 sec\n",
      " ARIMA(0,1,3)(0,0,2)[12] intercept   : AIC=947.471, Time=0.93 sec\n",
      " ARIMA(0,1,3)(1,0,0)[12] intercept   : AIC=602.141, Time=0.42 sec\n",
      " ARIMA(0,1,3)(1,0,1)[12] intercept   : AIC=399.079, Time=1.35 sec\n",
      " ARIMA(0,1,3)(2,0,0)[12] intercept   : AIC=500.296, Time=1.55 sec\n",
      " ARIMA(0,1,4)(0,0,0)[12] intercept   : AIC=1188.947, Time=0.19 sec\n",
      " ARIMA(0,1,4)(0,0,1)[12] intercept   : AIC=999.240, Time=0.55 sec\n",
      " ARIMA(0,1,4)(1,0,0)[12] intercept   : AIC=604.133, Time=0.50 sec\n",
      " ARIMA(0,1,5)(0,0,0)[12] intercept   : AIC=1091.452, Time=0.25 sec\n",
      " ARIMA(1,1,0)(0,0,0)[12] intercept   : AIC=1298.693, Time=0.05 sec\n",
      " ARIMA(1,1,0)(0,0,1)[12] intercept   : AIC=1075.553, Time=0.19 sec\n",
      " ARIMA(1,1,0)(0,0,2)[12] intercept   : AIC=971.074, Time=0.50 sec\n",
      " ARIMA(1,1,0)(0,0,3)[12] intercept   : AIC=882.846, Time=1.73 sec\n",
      " ARIMA(1,1,0)(0,0,4)[12] intercept   : AIC=818.711, Time=3.54 sec\n",
      " ARIMA(1,1,0)(1,0,0)[12] intercept   : AIC=inf, Time=0.34 sec\n",
      " ARIMA(1,1,0)(1,0,1)[12] intercept   : AIC=415.208, Time=0.60 sec\n",
      " ARIMA(1,1,0)(1,0,2)[12] intercept   : AIC=402.476, Time=2.12 sec\n",
      " ARIMA(1,1,0)(1,0,3)[12] intercept   : AIC=429.884, Time=4.39 sec\n",
      " ARIMA(1,1,0)(2,0,0)[12] intercept   : AIC=inf, Time=1.07 sec\n",
      " ARIMA(1,1,0)(2,0,1)[12] intercept   : AIC=419.269, Time=1.80 sec\n",
      " ARIMA(1,1,0)(2,0,2)[12] intercept   : AIC=409.187, Time=2.23 sec\n",
      " ARIMA(1,1,0)(3,0,0)[12] intercept   : AIC=inf, Time=2.84 sec\n",
      " ARIMA(1,1,0)(3,0,1)[12] intercept   : AIC=419.958, Time=4.93 sec\n",
      " ARIMA(1,1,0)(4,0,0)[12] intercept   : AIC=inf, Time=7.63 sec\n",
      " ARIMA(1,1,1)(0,0,0)[12] intercept   : AIC=1240.963, Time=0.07 sec\n",
      " ARIMA(1,1,1)(0,0,1)[12] intercept   : AIC=1069.162, Time=0.28 sec\n",
      " ARIMA(1,1,1)(0,0,2)[12] intercept   : AIC=973.065, Time=0.75 sec\n",
      " ARIMA(1,1,1)(0,0,3)[12] intercept   : AIC=884.323, Time=2.69 sec\n",
      " ARIMA(1,1,1)(1,0,0)[12] intercept   : AIC=588.156, Time=0.71 sec\n",
      " ARIMA(1,1,1)(1,0,1)[12] intercept   : AIC=399.034, Time=0.91 sec\n",
      " ARIMA(1,1,1)(1,0,2)[12] intercept   : AIC=409.611, Time=2.71 sec\n",
      " ARIMA(1,1,1)(2,0,0)[12] intercept   : AIC=503.551, Time=1.19 sec\n",
      " ARIMA(1,1,1)(2,0,1)[12] intercept   : AIC=399.928, Time=2.25 sec\n",
      " ARIMA(1,1,1)(3,0,0)[12] intercept   : AIC=457.277, Time=5.28 sec\n",
      " ARIMA(1,1,2)(0,0,0)[12] intercept   : AIC=1196.535, Time=0.10 sec\n",
      " ARIMA(1,1,2)(0,0,1)[12] intercept   : AIC=1042.432, Time=0.31 sec\n",
      " ARIMA(1,1,2)(0,0,2)[12] intercept   : AIC=948.444, Time=0.84 sec\n",
      " ARIMA(1,1,2)(1,0,0)[12] intercept   : AIC=591.273, Time=0.73 sec\n",
      " ARIMA(1,1,2)(1,0,1)[12] intercept   : AIC=400.256, Time=0.99 sec\n",
      " ARIMA(1,1,2)(2,0,0)[12] intercept   : AIC=501.159, Time=2.43 sec\n",
      " ARIMA(1,1,3)(0,0,0)[12] intercept   : AIC=1176.484, Time=0.15 sec\n",
      " ARIMA(1,1,3)(0,0,1)[12] intercept   : AIC=1039.309, Time=0.56 sec\n",
      " ARIMA(1,1,3)(1,0,0)[12] intercept   : AIC=604.131, Time=0.62 sec\n",
      " ARIMA(1,1,4)(0,0,0)[12] intercept   : AIC=inf, Time=0.54 sec\n",
      " ARIMA(2,1,0)(0,0,0)[12] intercept   : AIC=1180.404, Time=0.06 sec\n",
      " ARIMA(2,1,0)(0,0,1)[12] intercept   : AIC=1058.115, Time=0.21 sec\n",
      " ARIMA(2,1,0)(0,0,2)[12] intercept   : AIC=973.051, Time=0.64 sec\n",
      " ARIMA(2,1,0)(0,0,3)[12] intercept   : AIC=883.377, Time=1.65 sec\n",
      " ARIMA(2,1,0)(1,0,0)[12] intercept   : AIC=inf, Time=0.32 sec\n",
      " ARIMA(2,1,0)(1,0,1)[12] intercept   : AIC=405.142, Time=0.88 sec\n",
      " ARIMA(2,1,0)(1,0,2)[12] intercept   : AIC=426.092, Time=1.91 sec\n",
      " ARIMA(2,1,0)(2,0,0)[12] intercept   : AIC=inf, Time=1.38 sec\n",
      " ARIMA(2,1,0)(2,0,1)[12] intercept   : AIC=417.711, Time=2.47 sec\n",
      " ARIMA(2,1,0)(3,0,0)[12] intercept   : AIC=inf, Time=4.11 sec\n",
      " ARIMA(2,1,1)(0,0,0)[12] intercept   : AIC=990.719, Time=0.15 sec\n",
      " ARIMA(2,1,1)(0,0,1)[12] intercept   : AIC=881.526, Time=0.57 sec\n",
      " ARIMA(2,1,1)(0,0,2)[12] intercept   : AIC=837.402, Time=1.87 sec\n",
      " ARIMA(2,1,1)(1,0,0)[12] intercept   : AIC=588.171, Time=0.86 sec\n",
      " ARIMA(2,1,1)(1,0,1)[12] intercept   : AIC=443.647, Time=1.24 sec\n",
      " ARIMA(2,1,1)(2,0,0)[12] intercept   : AIC=501.151, Time=1.50 sec\n",
      " ARIMA(2,1,2)(0,0,0)[12] intercept   : AIC=988.094, Time=0.32 sec\n",
      " ARIMA(2,1,2)(0,0,1)[12] intercept   : AIC=757.716, Time=1.04 sec\n",
      " ARIMA(2,1,2)(1,0,0)[12] intercept   : AIC=595.040, Time=1.13 sec\n",
      " ARIMA(2,1,3)(0,0,0)[12] intercept   : AIC=1140.469, Time=0.28 sec\n",
      " ARIMA(3,1,0)(0,0,0)[12] intercept   : AIC=1126.139, Time=0.12 sec\n",
      " ARIMA(3,1,0)(0,0,1)[12] intercept   : AIC=996.923, Time=0.23 sec\n",
      " ARIMA(3,1,0)(0,0,2)[12] intercept   : AIC=918.438, Time=0.75 sec\n",
      " ARIMA(3,1,0)(1,0,0)[12] intercept   : AIC=inf, Time=0.40 sec\n",
      " ARIMA(3,1,0)(1,0,1)[12] intercept   : AIC=404.945, Time=0.98 sec\n",
      " ARIMA(3,1,0)(2,0,0)[12] intercept   : AIC=inf, Time=1.81 sec\n",
      " ARIMA(3,1,1)(0,0,0)[12] intercept   : AIC=989.496, Time=0.24 sec\n",
      " ARIMA(3,1,1)(0,0,1)[12] intercept   : AIC=856.486, Time=0.87 sec\n",
      " ARIMA(3,1,1)(1,0,0)[12] intercept   : AIC=604.951, Time=0.46 sec\n",
      " ARIMA(3,1,2)(0,0,0)[12] intercept   : AIC=991.558, Time=0.44 sec\n",
      " ARIMA(4,1,0)(0,0,0)[12] intercept   : AIC=1125.025, Time=0.09 sec\n",
      " ARIMA(4,1,0)(0,0,1)[12] intercept   : AIC=987.621, Time=0.26 sec\n",
      " ARIMA(4,1,0)(1,0,0)[12] intercept   : AIC=inf, Time=0.57 sec\n",
      " ARIMA(4,1,1)(0,0,0)[12] intercept   : AIC=988.660, Time=0.44 sec\n",
      " ARIMA(5,1,0)(0,0,0)[12] intercept   : AIC=1113.673, Time=0.11 sec\n",
      "\n",
      "Best model:  ARIMA(0,1,2)(1,0,1)[12] intercept\n",
      "Total fit time: 214.881 seconds\n"
     ]
    }
   ],
   "source": [
    "start_time = time.time()\n",
    "sarima_model = auto_arima(X_train_arima,\n",
    "                            start_p=2, d=None, start_q=1, max_p=10, max_d=10, max_q=10,\n",
    "                            start_P=2, D=None, start_Q=1, max_P=10, max_D=10, max_Q=10, m=12,\n",
    "                            suppress_warnings=True, stepwise=False, seasonal=True,\n",
    "                            error_action='ignore', trace=True, n_fits=50)\n",
    "sarima_time = time.time() - start_time\n",
    "autosarima_y_pred = sarima_model.predict(n_periods=12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Auto ARIMA Models MAPE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "auto arima mape = 0.00320610696849194\n",
      "auto sarima mape = 0.0007307187891033691\n"
     ]
    }
   ],
   "source": [
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print('auto arima mape', '=', sklearn_metric_loss_score('mape', y_test, autoarima_y_pred))\n",
    "print('auto sarima mape', '=', sklearn_metric_loss_score('mape', y_test, autosarima_y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare All"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "flaml mape = 0.0011216670337974744\n",
      "default prophet mape = 0.0011411103714832386\n",
      "auto arima mape = 0.00320610696849194\n",
      "auto sarima mape = 0.0007307187891033691\n"
     ]
    }
   ],
   "source": [
    "from flaml.ml import sklearn_metric_loss_score\n",
    "print('flaml mape', '=', sklearn_metric_loss_score('mape', y_test, flaml_y_pred))\n",
    "print('default prophet mape', '=', sklearn_metric_loss_score('mape', prophet_y_pred, y_test))\n",
    "print('auto arima mape', '=', sklearn_metric_loss_score('mape', y_test, autoarima_y_pred))\n",
    "print('auto sarima mape', '=', sklearn_metric_loss_score('mape', y_test, autosarima_y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAACB7UlEQVR4nOydZXRUVxeGnxMj7iGEeHCI4e7FSnF3KngLhRqVry11pVgp1iLFHQqU4u6uQQIJEULcbTJzvh8TAoEACWRi3GetuzJz7O6TTGbfY+8WUkoUFBQUFBQA9IrbAAUFBQWFkoPiFBQUFBQUclCcgoKCgoJCDopTUFBQUFDIQXEKCgoKCgo5GBS3AS+Cvb299PDwKG4zFBQUFEoVp0+fjpZSOuSVV6qdgoeHB6dOnSpuMxQUFBRKFUKI4CflKdNHCgoKCgo5KE5BQUFBQSEHnTkFIYSxEOKEEOK8EOKyEGJKdvpBIcS57CtcCLHxkXr1hRBZQojeurJNQUFBQSFvdLmmkAG0kVImCyEMgUNCiH+llM3vFxBCrAM2PfReH/gR2KFDuxQUygQqlYrQ0FDS09OL2xSFEoqxsTEuLi4YGhrmu47OnILUiiolZ781zL5yhJaEEJZAG+D1h6q9A6wD6uvKLgWFskJoaCgWFhZ4eHgghChucxRKGFJKYmJiCA0NxdPTM9/1dLqmIITQF0KcAyKBnVLK4w9ldwd2SykTs8s6Az2AP57R5kghxCkhxKmoqCjdGK6gUApIT0/Hzs5OcQgKeSKEwM7OrsAjSZ06BSmlWkrpD7gADYQQ3g9lDwBWPPR+GvCRlFLzjDbnSSnrSSnrOTjkuc1WQeGlQXEICk/jeT4fRbL7SEoZD+wFOgIIIeyBBsDWh4rVA1YKIYKA3sBsIUT3orBPoeSTqc7kn7PzuBx2tLhNUVAo0+hy95GDEMI6+7UJ0A4IyM7uDWyRUuaMa6SUnlJKDymlB7AWGCul3Kgr+xRKDwcC1tJjWWO+P7GMIdvH8cXGfsSkKFOHJYWNGzcihCAgIOCZZadNm0Zqaupz32vRokW8/fbb+U5/EXTRZmlAlyMFJ2CvEOICcBLtmsKW7Lz+5J46UlB4jJDYG7yzqiPf7FhKg4tvMujsF7xx/jPO3oYua9qw7PA3ZGmyitvMl54VK1bQrFkzVqx49r/0izoFBd2jM6cgpbwgpawtpfSVUnpLKb96KK+VlHL7U+oOl1Ku1ZVtCiWbNFUqs3aMZ9yST7A51YtuV8bjKmtRv7MHdnbOdLg2is7XhjH9yjb6LGvCicBtxW3yS0tycjKHDh3izz//ZOXKlTnparWa999/H29vb3x9fZk5cyYzZswgPDyc1q1b07p1awDMzc1z6qxdu5bhw4cD8M8//9CwYUNq167NK6+8wr179/JtU1RUFL169aJ+/frUr1+fw4cPo9Fo8PDwID4+PqdclSpVuHfvXp7lX2ZKtfaRQtlCSsnOi3+zdPdOPEPb0iHFDRNzDfX6VaVSw/J89981GnRxofLdTE5tE7x+oRZnnNfw5sGPaH/2D95vNwsnK/fi7kaxMOWfy1wJTyzUNmtWtOSLLrWeWmbTpk107NiRqlWrYmdnx+nTp6lbty7z5s0jKCiIc+fOYWBgQGxsLLa2tkydOpW9e/dib2//1HabNWvGsWPHEEKwYMECfvrpJ3799dd82T1hwgQmTpxIs2bNuHPnDh06dODq1at069aNDRs28Prrr3P8+HHc3d1xdHRk4MCBeZZ/WVGcgkKJIDDiAnNWzccyqBEN016nnHkaTQZVoVpjZ/T0BZPXXWTVqRCWHb/DhLZVGPZZffYtu4b/zcH4xzVjnccSum54jbfcOjG85deU0y9X3F16KVixYgUTJkwAoH///qxYsYK6deuya9cuRo8ejYGB9ivG1ta2QO2GhobSr18/7t69S2ZmZoH22e/atYsrV67kvE9MTCQ5OZl+/frx1Vdf8frrr7Ny5Ur69ev31PIvK4pTUChWElMT+HPJ76QFeOGR3gM9sxha9XemWvMq6OlrZzcXHwli1akQRrX0Ii4lk+m7bxAck8IP4/24eeweR9fr0+fS/4iqsJXZ8j82Lt3Fh/U/oFWN/i/Nls1nPdHrgtjYWPbs2cPFixcRQqBWqxFC8PPPP+e7jYf/Pg/vp3/nnXeYNGkSXbt2Zd++fXz55Zf5blOj0XDs2DGMjY1zpTdu3JibN28SFRXFxo0b+eyzz55a/mVFEcRTKBZUmVks/3sFcybvxOhcI0z1VDTrDqN/7k2NVtVyHMKRwGi+2nKFV2qU56MO1fmxly8fdKjGxnPhDP3rJM51HRj4ZSM8fMtjF/YaEwKmYB/nyPiT3zFmVXuCol/eaQBds3btWoYMGUJwcDBBQUGEhITg6enJwYMHadeuHXPnziUrS7sRIDY2FgALCwuSkpJy2nB0dOTq1atoNBo2bNiQk56QkICzszMAixcvLpBd7du3Z+bMmTnvz507B2gdUI8ePZg0aRI1atTAzs7uqeVfVhSnoFCkqDLU7Fh3iFnvbSXusCNqw3i8W11jws8D8OvYBqH34MkxJDaVccvO4Glvxm/9/NHTEwghGNe6MjMH1OZcaDw9Zh8mMiuLTqN86DTaBz3saXztA965M5TLydH02NKHqdtHk5L58k4H6IoVK1bQo0ePXGm9evVixYoVvPXWW7i5ueHr64ufnx/Lly8HYOTIkXTs2DFnofmHH37gtddeo0mTJjg5OeW08+WXX9KnTx/q1q37zPWHR5kxYwanTp3C19eXmjVrMmfOnJy8fv36sXTp0pypo2eVfxkRWomi0km9evWkEmSndJCRlsXJnTc4u/M2eipj7lleo2aV6/QZ8An65o+fTE/JyKLXH0cIj09j09vN8LQ3e6zM6eBYRiw5jUZK5g2pRwNPWzLSsji2IZBLB8IwtZSEOv7JavuLOEh9JvmNobP/yDIzpXT16lVq1KhR3GYolHDy+pwIIU5LKevlVV4ZKSjolLTkTI5uusmfH+3h/La7hJjdJKXmH7z3tj/9R/yWp0PQaCTvrT7P9XtJzBpYJ0+HAFDX3ZaNY5tiZ2bE4AXH2XA2lHImBrQcWI0e79ehnKkZtjfe4n/hn+GcZsTHF2YxfEVLAsJP6rrbCgqlFmWhWUEnpCRkcG7nHS7sD0Gt0nDL9hLpjv8xoWFnqjVcDU95Wp+55ybbL0fwWecatKj6dH0rNztT1o9pyuilp5m46jzBMalMaFuFipWt6fdpA05vD+L0dkHLqJ/p4LGTeWbr6bfjdfrY1ebtV6ZjbVKwXTEKCmUdxSkoFCpJsemc/S+Yy4fDUWepuWF/muAK/zHS1Y1O7VcjTKyeWv+/yxH8tus6PWs782az/G1DtDI1ZPEbDfhkw0Wm7bpBcEwqP/TyoZyhPg26eFG5riN7lwYQcaUNEzxe4YbVT6yMOcv2Va0YX30IvRpMQl9PvzC6r6BQ6lHWFBQKhfh7qZz5L5iAYxFI1NywP8aZirvobpLFqPazMHPyf2Yb1yKS6Dn7MJXLm7NqVGOMDQv2RS2lZPa+QH7+7xoNPG2ZO7guNmZG2jyN5PLBMI5sCESqJZ6+4ayQ33OqHFTXM+Pj5t9Sx6Pt83S92FDWFBTyQ0HXFBSnoPBCxIQnc/rfYG6eugf6kmCHI+x32oGfXhyTa0/Es84bT50quk9cSiZdfz9EhkrD5rebUcHq+feMbz4fzvtrzuNsbcLC4fXxeGhNIjkunQMrr3P7fDT2LqYYu61hpupf7hno09miCpPazaK8RcXnvndRojgFhfygLDQrFBmBZyJZ+fUJbp+PJN79FIv8/sd5lxV8W9GbOUOO41n3zXw5hCy1hrdXnOFeQgZzhtR9IYcA0NWvIsvfakhCmooesw9zMig2J8/cxphOo33oONKb1MQswo525jODPxmhLs+OxOt0WdeBv/Z/hkqteiEbFBRKK4pTUHguUhIy2LssAGGTwFKfyax3XMzrJiZs7Lqetq/9gSiX946hvPh221UO34zh2x7e1HGzKRT76nnYsmFsE2zMjBg0/zibzoXl5AkhqFSnPAO/bEiNZhW5fDQNm8CvmW/3Pxqo4LegTfRc1phD1zY85Q4KAPr6+vj7++dcQUFB7Nu3j9dee+2Jdfz9/enfv3+utOHDh2NqaprrYNu7776LEILo6Gggt3heXmRkZPDKK6/g7+/PqlWrXqBXhcd3331X3CYUGMUpKBQYKSX7lgaQnpbOcrdZ1BFpbKz7GWMG78bYoXqB2lp9KoSFh4N4o6knfeq5Fqqd7nZmrB/ThNpu1kxYeY4Zu2/w8HRpOVNDWg+qTo/3aqNnoMexrQ501VvGDKuOyMwUxhz7nG82Dy5Um8oaJiYmnDt3Lufy8PB4avmrV6+iVqs5ePAgKSkpufIqV67Mpk2bAK30xJ49e3JONeeHs2fPAtoTyQ8fTnsaarU63+0/D4pTUHgpuHrkLkEXYzjitpl2JmlMH34SF98BBW7ndHAcn224RNPKdnzyasGcSX6xNjXi7zcb0rOOM1N3Xue9NefJyMr9RVCxig39PqtPvVc9uHk6hlsHu/Kj50r6S0tWxZ3nyJWS8dRZFlixYgVDhgyhffv2OQ7gPv379895wt+3bx9NmzbNEdR7FpGRkQwePJiTJ0/i7+9PYGAgu3fvpnbt2vj4+PDGG2+QkZEBgIeHBx999BF16tRhzZo17Nixg8aNG1OnTh369OmTI4Z38uRJmjRpgp+fHw0aNCApKYmgoCCaN29OnTp1qFOnDkeOHAHg7t27tGjRAn9/f7y9vTl48CCTJ08mLS0Nf39/Bg0aVFi/Qp2jbElVKBCJMWkcXH2dSMsbpNrv48OuW8Cg4IqkEQnpjF56mgpWxswaUAcDfd09nxgZ6PFrHz887cz4ded1wuLSmDukLtamRjllDAz1adjVi8p1y7N3aQD71sTgXXUqNYzf59sTP7C+ajfKGZRgwbR/J0PExcJts4IPdPrhqUXuf+kBeHp65tIvyotVq1axc+dOAgICmDlzJgMHDszJq1q1Kps3byYuLo4VK1YwePBg/v3333yZWr58eRYsWMAvv/zCli1bSE9Pp1WrVuzevZuqVasydOhQ/vjjD959910A7OzsOHPmDNHR0fTs2ZNdu3ZhZmbGjz/+yNSpU5k8eTL9+vVj1apV1K9fn8TERExMTChfvjw7d+7E2NiYGzduMGDAAE6dOsXy5cvp0KEDn376KWq1mtTUVJo3b86sWbNKnZaSMlJQyDdSI9m9+CoZWWns81rGj3Xfw8TGo8DtpKvUjPr7FKkZWSwYVi9n26guEULwTtsqTO/vz9k78fScfYTgmJTHytk5m9Pzg7o071eVe8HptL81mVAp+XPnuzq3sTTy8PTRsxzCqVOnsLe3x83NjbZt23L27Nkcobz79OzZk5UrV3L8+HGaN2/+3HZdu3YNT09PqlatCsCwYcM4cOBATv796aVjx45x5coVmjZtir+/P4sXLyY4OJhr167h5ORE/fr1AbC0tMTAwACVSsWIESPw8fGhT58+OZLb9evXZ+HChXz55ZdcvHgRCwuL57a9uFFGCgr55uL+UMKvx3PQaz2jHCpSrfbrBW5DSsnH6y9yPjSBeUPqUtWxaP95uvk7U9HahJFLTtH998PMH1qPeh65TzXr6Ql8W7tgVd6ELTPP0+duGxaIHbx67zwejn5Fam++ecYTfUlgxYoVBAQE5Kw7JCYmsm7dOkaMGJFTpl+/ftStW5dhw4ahp6e7Z1YzM+1GCCkl7dq1eyyU6MWLeY+6fvvtNxwdHTl//jwajSZHbrtFixYcOHCArVu3Mnz4cCZNmsTQoUN1Zr8uUUYKCvki/l4qh9fd4I71JSrYnmJA10XP1c6fh26z4WwYk9pVpX2tCoVrZD6p72HLhrFNsTY1YuAjO5Mexr2WHW61bCkf+RqWmSZ8s+ttSvO5nuJEo9GwevVqLl68SFBQEEFBQWzatOmxL2N3d3e+/fZbxo4d+0L3q1atGkFBQdy8eROAv//+m5YtWz5WrlGjRhw+fDinXEpKCtevX6datWrcvXuXkye1OllJSUlkZWWRkJCAk5MTenp6/P333zkL1cHBwTg6OjJixAjeeustzpw5A4ChoSEqVena3qw4BYVnolFr2LHwIukylUsey/mq03yEYcHn1/dfj+K7bVfp5F2Bt1tX1oGl+cfDXrszyT97Z9LMR3Ym3adJr8pkZQqGRA/jeFY8W09OLwZrSx+7d+/GxcUl5zp48CDOzs5UrPjgYGCLFi24cuUKd+/ezVV31KhRVKpU6bE2U1NTc7U5derUJ97f2NiYhQsX0qdPH3x8fNDT02P06NGPlXNwcGDRokUMGDAAX19fGjduTEBAAEZGRqxatYp33nkHPz8/2rVrR3p6OmPHjmXx4sX4+fkREBCQM+LYt28ffn5+1K5dm1WrVuVEoxs5ciS+vr6laqFZOdGs8ExObw/i2MZb7Kq8iMn1q9Cw5f8K3Mbt6BS6zTpERWsT1o1pglm5kjFzmZGlZvK6i2w4G0avOi5839MHI4Pcz0r7l1/j8qEwztT8hhtmUWzutw8rU7tisvgByolmhfygnGhWKFSiQ5M5tjmQm3ZnaOUUTsMWnxW4jaR0FSOWnEJfTzB/aL0S4xAAyhnoM7WvH+++UoV1Z0IZ+tdxElJzD/cbdPHE0EifVxInES8kM/57/IlTQaGsoDgFhSeiztLw74KzpOknEeu6hrHdV+RLtuJhNBrJuyvPcTs6hd8H1cHV1lRH1j4/QgjefaUqv/Xz40xwPD3+OJxrZ5KJhRF1X/Ug5o4pQ5IasCbhKudv7ShGixUUdIfiFBSeyPEtgSRGqDjmuYJvWn+GodnTYxvkxa87r7E7IJIvutSkSaWChVUsanrUduHvNxsQm5JJj9lHOB38YLukX2tXLO2NqRg9HIcsydeHPiVLk1WM1ioo6AadOQUhhLEQ4oQQ4rwQ4rIQYkp2+kEhxLnsK1wIsTE7vZsQ4kJ2+ikhRDNd2abwbCJuJ3Bm+x0CHI7xRjUbXKt1KXAb/5wP5/e9gfSv78qQRu46sLLwaehlx4axTbE0NmDA/ONsv6RdBNU31KNxj8rE3ctijHoE12Q6y/cXfCpNQaGko8uRQgbQRkrpB/gDHYUQjaSUzaWU/lJKf+AosD67/G7ALzv9DWCBDm1TeAqqTDVb550k2SgOa7ftvNrp9wK3cTk8gQ/Wnqeeuw1fdfMuVXGRPe3N2DC2KTUqWPDRuoskZ2hHBJXqOOBU2YrEm3VplWnGrOAtRMQGFrO1CgqFi86cgtSSnP3WMPvK2eokhLAE2gAbs8snywdbocweLqtQtOxfc4n0OD2uei7l4x4LoIBRyaKTMxi55DQ2pkb8MbjuY7t5SgM2ZkZ80bUWCWkqVhy/A2jXHpr2rkJakoouelOQUvLjzjHFbKmCQuGi0/9WIYS+EOIcEAnslFIefyi7O7BbSpn4UPkeQogAYCva0UJebY7Mnl46FRUVpTvjX1JCA2K5djCGy477+bBZB0ztqxaofmaWhrFLzxCdnMG8IfVwsCi4LlJJoY6bDY297Jh/8FaOiJ6jhyXVGlbg1tlyjNSrza70u+y/sKSYLS0+7ktne3t706dPH1JTU1+4zaCgILy9vQtUZ+PGjTmSE48SFRVFw4YNqV27NgcPHnxh+16U+Ph4Zs+eXdxmPBGdOgUppTp7OsgFaCCEePgvPQBY8Uj5DVLK6mgdxtdPaHOelLKelLKeg0PBFz4VnkxmWhb/zD9GvHEkDapfoVb9gj8FT/nnMieCYvmpty8+Lk+Px1waGNu6EpFJGaw7/eDUc6PuXggBLmnvUilLw3dnfiUt83EdpZeB+9pHly5dwsjIiDlz5uTKz8oqmsX4pzmF3bt34+Pjw9mzZ/Otp6RLSe2X2incR0oZD+wFOgIIIeyBBmhHBHmVPwB4ZZdTKCK2LTlCVooRMZ7LGdpzYYHrLz0WzLLjdxjV0otu/vnXwS/JNKtsj6+LFXMPBJKl1gDa6G3+7d24dS6RCbZjCBca5u4cX8yWFj/Nmzfn5s2b7Nu3j+bNm9O1a1dq1qxJeno6r7/+Oj4+PtSuXZu9e/cCsGjRIrp160arVq2oUqUKU6ZMyWlLrVYzYsQIatWqRfv27UlLSwMgMDCQjh07UrduXZo3b05AQABHjhxh8+bNfPDBBzmy2fc5d+4cH374IZs2bcLf35+0tDRWrFiBj48P3t7efPTRRzllzc3Nee+99/Dz8+Po0aMsXbqUBg0a4O/vz6hRo3Icxfbt26lTpw5+fn60bauN633ixAkaN25M7dq1adKkCdeuXQPg8uXLOW34+vpy48YNJk+eTGBgIP7+/nzwwQe6/aM8Bzo7RSSEcABUUsp4IYQJ0A74MTu7N7BFSpn+UPnKQKCUUgoh6gDlgBhd2aeQm+tnwwk7m8U1p1182f1D9Mo9PcrVo5y4HcuXmy/TqpoDH3bQTWyE4kAIwdhWlRi99AzbLkXQ1U8r01C7nRtXDoUTc60R3R2XsTjqOK+Fn6ByxQbFYuePJ34kIDagUNusbludjxp89OyCaEcE//77Lx07dgTgzJkzXLp0CU9PT3799VeEEFy8eJGAgADat2/P9evXAe2X6aVLlzA1NaV+/fp07twZe3t7bty4wYoVK5g/fz59+/Zl3bp1DB48mJEjRzJnzhyqVKnC8ePHGTt2LHv27KFr16689tpr9O7dO5dd/v7+fPXVV5w6dYpZs2YRHh7ORx99xOnTp7GxsaF9+/Zs3LiR7t27k5KSQsOGDfn111+5evUqP/74I4cPH8bQ0JCxY8eybNkyOnXqxIgRIzhw4ACenp45Kq/Vq1fn4MGDGBgYsGvXLj755BPWrVvHnDlzmDBhAoMGDSIzMxO1Ws0PP/zApUuXSqykti6PljoBi4UQ+mhHJKullFuy8/oDj8o69gKGCiFUQBrQ76GFZwUdkp6s4r9FZ4gzjadvYwPs3Qu2Gzg0LpUxS0/jZmfK9P610dcrPTuN8kP7mhWo5GDG7L036eLrhBACI2MDGnWrxJ4lV+lZ+1f2ho3g6z0TWTjoIHqi9C2sPy8Px1No3rw5b775JkeOHKFBgwZ4enoCcOjQId555x1A++Xp7u6e4xTatWuHnZ1WMqRnz54cOnSI7t274+npmdNu3bp1CQoKIjk5mSNHjtCnT5+c+98PnJNfTp48SatWrbg/9Txo0CAOHDhA9+7d0dfXp1evXoB2yun06dM50tlpaWmUL1+eY8eO0aJFi5y+2dpqFXYTEhIYNmwYN27cQAiRI4LXuHFjvv32W0JDQ+nZsydVqlQpkL3Fgc6cgpTyAlD7CXmt8kj7kQcjCYUiZM2cf5GZJpj6/kOLdisLVDctU83IJafJzNIwf2g9rEwMdWRl8aGnJxjTqjLvrznP3muRtKnuCED1RhW4sDeEC/tUTGrQmC8Sj7Pp2M/0aJy/p+vCJL9P9IXN/TWFR7kvFPcsHt2qfP99uXIPNijo6+uTlpaGRqPB2tpaZ0/YxsbG6Otrd9pJKRk2bBjff/99rjL//PNPnnX/97//0bp1azZs2EBQUBCtWrUCYODAgTRs2JCtW7fy6quvMnfuXLy8vHRif2Hx8jzSKOTJqQMXSbxpzh2nfxk/aFqBZCyklHyw9jxXIxKZMaA2lRwKNuVUmujmXxFnaxN+3xuYo6Yq9ATN+lQhOS4DV8MPqZMFUwOWEpd8r5itLVk0b96cZcuWAXD9+nXu3LlDtWrVANi5cyexsbGkpaWxceNGmjZt+sR2LC0t8fT0ZM2aNYD283f+/HkALCwsSEpKeqYtDRo0YP/+/URHR6NWq1mxYkWektpt27Zl7dq1REZGAhAbG0twcDCNGjXiwIED3L59OycdtCOF+/GkFy1alNPOrVu38PLyYvz48XTr1o0LFy7k29biQnEKLzFJcWkcXh1EtFkwY7s3wsjSKd91M7LUvL/mAlsu3OXDDtVpXb28Di0tfgz19RjZwovTwXGcuP1A/sK5qg1e/g6c3XmXD2tOJllIflME83IxduxYNBoNPj4+9OvXj0WLFuWMBBo0aECvXr3w9fWlV69e1KuXp3BnDsuWLePPP//Ez8+PWrVq5cR57t+/Pz///DO1a9fOtdD8KE5OTvzwww+0bt0aPz8/6tatS7du3R4rV7NmTb755hvat2+Pr68v7dq14+7duzg4ODBv3jx69uyJn59fTgS3Dz/8kI8//pjatWvn2nG1evVqvL298ff359KlSwwdOhQ7OzuaNm2Kt7d3iVxoVqSzX1KklPzx1QpUEbZUbLaNPoNm5LtuVFIGo/4+xZk78Ux8pSrj21YuVSeWn5e0TDXNftyDt7MVi994sKAcH5nKiinHqdaoAhf03uHPrHssavI9dau8plN7Srt09qJFi3IWgBV0hyKdrZAvdmzeibxbgUTX/+jd75d817scnkC3WYe4cjeR2YPqMOGVKi+FQwAwMdLnjWae7L8exaWwhJx06/Km+LR24eqRu/T2n4VzlppvjnyBKiuzGK1VUHg+FKfwEnI3/B5Xd6iItrjBhNffQBgY5ave9ksR9P7jKBJYO7oJr/rkf7qprDCksTsW5Qz4Y1/uKYp6nTwwNjXk1I40Pnbryk0yWbJvcjFZWToYPny4MkoogShO4SVDo9awbOY2pBR0bJeFpZPPM+tIKZm15wajl56mupMFm95uirdz6T+t/DxYGhsyuLE72y7dJTAqOSfd2MyQ+q95EnYtDjeHibyiNmRO6A7CYq4Vo7UKCgVHcQovGcuXLMEkzh2Tygdo2H7cM8unq9SMX3mOX3Zcp0dtZ1aMaER5i4LHZy5LvNHUEyN9Pebuzz1aqNWiIjYVTDmy4TYfNPsRPSn5bsfYPGM/KyiUVBSn8BJx6cppYk5WIMHqCm+Nfva+9nuJ6fSde5QtF8L5qGN1pvb1w9iwYIqpZREHi3L0q+/KhrNhhMen5aTr6+vRpFdl4u+lEhVanXEWNTiQGcme838Wo7UKCgVDcQovCemZ6WxeeAG1UDFwYHUMTG2eWv5CaDxdZx0iMDKZeUPqMaZVpZdmQTk/jGzhhZQw/+CtXOnu3na4VLfh5Jbb9Gw5i2oqDd+dm0lKRuITWlJQKFkoTuEl4Y/ZM7FKcsez9jU8/F55atl/zofTZ85RDPX1WDe2Ce1qOhaRlaUHFxtTuvpXZOWJEGKSH0gtCKE90JaZlsXZvYl87j2CKNT8vuPZU3WllY0bNyKEICAgf9pL06ZNy7fE9rlz5xBCsH379qeWe/XVV4mPj89XmwXhgw8+oFatWiXmPMGiRYsIDw/X6T0Up/ASsHP/GvSv+ZFpe4nur098YjmNRjJ1xzXeWXEWPxdrNo1rSvUKlkVoaeliTMtKpKnULDoSlCvdztmcGs0qcmlfGG4eI+gjLFkWc5arIYeKx1Ads2LFCpo1a8aKFSueXZiCOYVntS2lRKPRsG3bNqytrfNrcr6ZN28eFy5c4Oeff85XeV1LhStOQeGFuRcbwvFNaWTppzJ8bBeEft5rAqmZWYxddoYZe27St54LS99qiJ156Q2QUxRUcbSgQy1HFh8JIildlSuvYRcv9I30OLw+kPHt/8Bao+Hrfe+j1uhOp784SE5O5tChQ/z555+sXPlAN2vfvn289tqDw3tvv/02ixYtYsaMGYSHh9O6dWtat24N8EQpaykla9asYdGiRezcuZP0dK2oclBQENWqVWPo0KF4e3sTEhKCh4cH0dHRBAUFUb16dYYPH07VqlUZNGgQu3btomnTplSpUoUTJ04AT5a6fpiuXbuSnJxM3bp1WbVqFUFBQbRp0wZfX1/atm3LnTvaiHzDhw9n9OjRNGzYkA8//DBPeW+Ae/fu0aNHD/z8/PDz8+PIkSMAdO/enbp161KrVi3mzZsHaKXDhw8fjre3Nz4+Pvz222+sXbuWU6dOMWjQoBwZcF2gS5VUhWJGIzX8PudPHFNbULtjNDYunnmWC4tPY8TiUwREJPK/12ryRlMPZf0gn4xtVZn/Lt9j+fE7jGpZKSfd1NKIuh3dObbxFn5t/PmgfHM+jjnCuiPf0rfZ54VuR8R335FxtXCls8vVqE6FTz55aplNmzbRsWNHqlatip2dHadPn6Zu3bpPLD9+/HimTp3K3r17sbe3f6qU9ZEjR/D09KRSpUq0atWKrVu35qiY3rhxg8WLF9OoUaPH7nHz5k3WrFnDX3/9Rf369Vm+fDmHDh1i8+bNfPfdd2zcuPGJUtcPs3nzZszNzXME+Lp06cKwYcMYNmwYf/31F+PHj2fjxo0AhIaGcuTIEfT19Wnbtm2e8t7jx4+nZcuWbNiwAbVaTXKydkvzX3/9ha2tLWlpadSvX59evXoRFBREWFgYly5dArSBeaytrZk1axa//PLLM+VAXgRlpFCGWbTuBxzuNKVcxes06d43zzKng+PoNuswIbGp/DW8Pm8281QcQgHwc7WmWWV7Fhy6Tboq9yjAr60rFrbGHFp7k06v/EbDLMG0m2uITgx7QmuljxUrVtC/f39Aqz+U3ymk+zwsZW1gYJAjZf2stt3d3fN0CACenp74+Pigp6dHrVq1aNu2LUIIfHx8CAoKArQCdn369MHb25uJEydy+fLlZ9p69OhRBg4cCMCQIUM4dOjBdGCfPn3Q19fPJe99PzjP3bt3AdizZw9jxmijGerr62NlpT3rM2PGDPz8/GjUqBEhISHcuHEDLy8vbt26xTvvvMP27duxtCy6aVxlpFBGuXBzH3cPeWBmlMjgdwfnWWb9mVAmr7uIk7UxK0c2pHJ5iyK2smwwtlUlBi44ztrToQxu5J6TbmCoT+Oeldix4DLXTsfzaYNP6HX6G37ZMZofeuctwfy8POuJXhfExsayZ88eLl68iBACtVqNEIKff/4ZAwMDNBpNTtn7Uz/5Ra1Ws27dOjZt2sS3336LlJKYmJgcddGnSXM/LLutp6eX815PTy9nzv9JUtfPy317CirvvW/fPnbt2sXRo0cxNTWlVatWpKenY2Njw/nz5/nvv/+YM2cOq1ev5q+//nohG/OLMlIog6RkJLJk2V6s0yvQsY8Lxpa5Ja3VGskP/wYwafV56nnYsHFsU8UhvACNK9nh52qdK2TnfSrXLU8FL0uOb7qFc5XevGnkwtaUII5dW19M1hYea9euZciQIQQHBxMUFERISAienp4cPHgQd3d3rly5QkZGBvHx8ezevTun3sPS0U+Sst69eze+vr6EhIQQFBREcHAwvXr1YsOGDYVi+5Okrp9GkyZNctZNli1blme856fJe7dt25Y//vgD0Dq9hIQEEhISsLGxwdTUlICAAI4dOwZAdHQ0Go2GXr168c0333DmzBkg/xLhL4LiFMogvyz5GI+7zSlfJYLqzRvmykvOyGLU36eYsz+QwY3cWPxGA2zM8qd9pJA3QgjGtapESGwaWy7cfSyvaZ8qpCZmcua/YN56dT6uWWq+OfYNGVkFe3ouaaxYsYIePXrkSuvVqxcrVqzA1dWVvn374u3tTd++fald+0G8rZEjR9KxY0dat279RCnrp7VdGDxJ6vppzJw5k4ULF+Lr68vff//N9OnT8yz3JHnv6dOns3fvXnx8fKhbty5XrlyhY8eOZGVlUaNGDSZPnpwzJRYWFkarVq3w9/dn8ODBOcF+7i9q63KhWZHOLmP8c+R3Lq0qj4XQY9QP3TA0fjBDGBKbyluLT3EzKpkvu9RkSGOP4jO0jKHRSDpMO4CeEPw7oTl6j4Qk3fHnZW6di2LQlEZcPPcNo0I2MdapFWPaz3zue5Z26WyFokGRzn6JCYm+wu7tcVhk2ND9rXq5HMLxWzF0nXWIiMR0lrzRQHEIhYyenmBs60pcu5fE7oDIx/Ib99DuTDq6IZAmrabQSW3EgvC9BEdeLGpTFRSeiuIUygiqrEx+WPYjVSKbULWeBhfvBwueK0/cYdCC49iYGbFxXFOaVrYvRkvLLl18K+JiY8LsfTcfE8GzsDXG/xVXbpy8R0RwMh+0/hUjKfl21zuKYJ5CiUJxCmWEGevep9L1PhhZxtNmWFsAstQavvrnCpPXX6RJZXs2jG2Kp33+AqorFBwDfT1GtazE2TvxHLsV+1h+nQ7umFoacXjNTew9WjLe0oejqhi2n55dDNYqKOSN4hTKAHtPLyftaGMMhaTHgFroG+iRkKbijcWn+Ovwbd5o6slfw+phZWJY3KaWefrUdcHevByz9918LM/I2ICG3byIuJXAzdOR9O30B7VUGn66NJfEtLhisFZB4XEUp1DKCY+5xb4Vcdik2VP/8t9EDehOQLPmbOzzFsa7/+WXVk583qUmBvrKn7ooMDbU581mnhy8Ec2F0PjH8qs3dsLOxZyjGwKRhpb8z3cssWiYuWNs0RuroJAHyjdFKSb9bhhbPlpM+eQaVLu+EhfvCiSPfY/D5u5UC7nCxNMrqfXuIAJfe42Ib78jae9e1MkpxW12mWdwIzcsjA2YvTfwsTw9PUHT3pVJiknnwp5QatUfywA9G1bFXeRS8N5isFZBITc6cwpCCGMhxAkhxHkhxGUhxJTs9INCiHPZV7gQYmN2+iAhxAUhxEUhxBEhhJ+ubCvtZN65w93Pv+DQoP+hMmhJ+YxjNJ//MYeGvE//iIqs7jiSirv34rlxA+U/+ABDxwrEr15N6JixXG/UiKBBg4ma9TupZ84iVapn31ChQFgYGzKssQf/XYngZmTyY/mu1W3x8LXn1L9BpCapeLvDHOw1Gr7a/1GpFMzTlXT2X3/9hY+PD76+vnh7e+fs988vmzdv5ocffihQnfwQEBCAv78/tWvXJjDwccdf1AQFBbF8+fLCa1BKqZMLEIB59mtD4DjQ6JEy64Ch2a+bADbZrzsBx591j7p168qXibRr12Toe+/LKzVqysONXpMzR/4np038Q6ozMuW8/YHS/aMtcuifx2ViWuZjddXp6TL56FF579ep8lav3vJK9RrySrXqMqBOXXlnzFgZ8/dSmR4YKDUaTTH0rOwRnZQuq322Tb63+lye+XERKXL2mD1yz9KrUkopt24ZI70Xecudp/7I9z2uXLlSKLa+KH379pXNmjWTn3/+eb7Ku7u7y6ioqKeWCQkJkV5eXjI+Pl5KKWVSUpK8detWvm1SqVT5LltQvv/+e/n111/nu7xGo5FqtVpn9uzdu1d27tz5ifl5fU6AU/JJ391PyijMCzAFzgANH0qzBOIAyzzK2wBhz2r3ZXEKqefPyztjx8kr1arLq7XryCtffCt/HbtR/jh+hYy5c1X++l+AdP9oixy77LTMUOXvw6eKjZUJ/26X4f/7XN54pZ28Uq26vFKturzespUM+/gTGb/5H6l6xj+uwtP5YtMlWenjrTI0LjXP/AOrrsnfR++W0aFJMis1TnZcUEMOWtIw3+2XBKeQlJQkK1asKK9duyarVq2ak/7oF9W4cePkwoUL5fTp06WhoaH09vaWrVq1klJKuXz5cunt7S1r1aolP/zwQymllKdPn5Z+fn4yKyvrsXvOmzdP1qtXT/r6+sqePXvKlJQUKaWUw4YNk6NGjZINGjSQEydOlAsXLpTjxo3LyRs9erRs2LCh9PT0lHv37pWvv/66rF69uhw2bFhO26NHj5Z169aVNWvWzNPJbd26VTo6OsqKFSvm2P/rr7/KWrVqyVq1asnffvtNSinl7du3ZdWqVeWQIUNkzZo1ZVBQkPzpp59kvXr1pI+PT662Fy9eLH18fKSvr68cPHiwlFLKzZs3ywYNGkh/f3/Ztm1bGRERIaWUct++fdLPz0/6+flJf39/mZiYKBs2bCgtLS2ln5+fnDp16mM2F9Qp6FQQTwihD5wGKgO/SymPP5TdHdgtpcwrTuGbwL9PaHMkMBLAzc2tUO0tSUgpST1+gph5c0k5chQ9Kyvsx43DrHc/Zv+yEz30eaVrJjPPqVl4OIh+9Vz5rqcP+nr5Uzg1sLHBsmMHLDt2ACAzJISUI0dJOXKEpN27SViv1eYpV60aZk2aYNakMab16qFnYqKzPpc1RrTwYumxYOYfuMWXXWs9ll+/syfXjkVweO0Nuoz3Z6hdXb5LOMe5m1vxr9y5QPc6uPo60SGPT1W9CPau5jTvW/WpZXQlnd2lSxccHR3x9PSkbdu29OzZky5dugDQs2dPRowYAcBnn33Gn3/+yTvvvAPklrB+VNMoLi6Oo0ePsnnzZrp27crhw4dZsGAB9evX59y5c/j7+/Ptt99ia2uLWq2mbdu2XLhwAV9f35w2Xn31VUaPHo25uTnvv/8+p0+fZuHChRw/fhwpJQ0bNqRly5bY2NjkkvfesWMHN27c4MSJE0gp6dq1KwcOHMDOzo5vvvmGI0eOYG9vT2ysditzs2bNOHbsGEIIFixYwE8//cSvv/7KL7/8wu+//07Tpk1JTk7G2NiYH374gV9++YUtW7YU+G+cFzpdaJZSqqWU/oAL0EAI4f1Q9gDgMSETIURrtE4hz8jyUsp5Usp6Usp6Dg4OOrC6eJFSkrRnL8H9B3Bn+HDSr9+g/AfvU3n3buzHjWPxgp2USylPeb/9rIzzY+HhIN5o6skPvfLvEPLCyNUVm359cZk+japHDuOxZg0OEyeib21N3NKlhIwYyfUGDQkeOozoOXNJu3gRqS59899FibO1CT1qO7PixB2iHwrZeR9jM0Pqd/Yk5GocwZdi6NZiClZqDYtO/lYM1j4fupLO1tfXZ/v27axdu5aqVasyceJEvvzySwAuXbpE8+bN8fHxYdmyZblkr+9LWOdFly5dciS0HR0dc8lr35fUXr16NXXq1KF27dpcvnyZK1euPNX+Q4cO0aNHD8zMzDA3N6dnz54cPHgQyC3vvWPHDnbs2EHt2rWpU6cOAQEB3Lhxgz179tCnTx/s7bUHSm1tbQGtc+vQoQM+Pj78/PPPOX1s2rQpkyZNYsaMGcTHx2NgUPjP9UUinS2ljBdC7AU6ApeEEPZAAyCX4pUQwhdYAHSSUsYUhW0lBalWk7h9OzHz5pNx7RqGFStS4YvPserZE71s6d9Ni3eiCatAnOtmbhj3Y/OpUCa0rcK7r1Qp1BgIQl8fEx9vTHy8sR81Ek1aGqmnTpNyVDuSiJo2jahp09CzssK8aRPtYraTU6HdvywxulUl1p4JZeHh23zQofpj+d4tnbm4P5Qj627S738N6Gfqzvz0OwTfu4C7o28eLebNs57odYEupbNBKybYoEEDGjRoQLt27Xj99df58ssvGT58OBs3bsTPz49Fixaxb9++nDr5kdR+WE77/vusrCxu377NL7/8wsmTJ7GxsWH48OHPZXdetkgp+fjjjxk1alSuMjNn5q199c477zBp0iS6du3Kvn37chzi5MmT6dy5M9u2baNp06b8999/z23fk9Dl7iMHIYR19msToB1wf3tCb2CLlDL9ofJuwHpgiJTyuq7sKmnIzEzi164l8NVXCX/vfaRKhdMP31Ppv+3YDBiQ4xAuHLlJ6FF9QuyPEWffkM1XYvmscw0mtquq86A4eiYmmDdvhuOHH+C1cQNVDh+i4i+/YNG2Lcn79nO7V29SssMcKuSmkoM5nbwrsORoMInpj+/00jfQo2mvysRFpHLlYDgDmnyGAfD34SlFb2wB0aV0dnh4eI5cNMC5c+dwd9dKtyQlJeHk5IRKpWLZsmWF1p/ExETMzMywsrLi3r17/PtvnjPYuWjevDkbN24kNTWVlJQUNmzYkKekdocOHfjrr79yoq2FhYURGRlJmzZtWLNmDTEx2mfg+9NHD0t7L168OKedwMBAfHx8+Oijj6hfvz4BAQGFLqety+kjJ2CvEOICcBLYKaW8P+nVn8enjj4H7IDZ2dtVy7T8qSYtjdglf3OzfQfufvY/9M3McZ4xHa8t/2DdvTvC8MHp48jgRPYvDeSe+S30HeLZGGTJ9z19eKu5V7HYbmBnh9Vrnan43bd4rFmNvpUVd15/g9glSxQdnzwY26oySelZLD0WnGe+h689ztWsOfHPbSzLN6CLsGJj4jVikyOK2NKCoUvpbJVKxfvvv0/16tXx9/dn1apVOVLVX3/9NQ0bNqRp06ZUr/746Ot58fPzo3bt2lSvXp2BAwfStGnTZ9apU6cOw4cPp0GDBjRs2JC33norV1/v0759ewYOHEjjxo3x8fGhd+/eJCUlUatWLT799FNatmyJn58fkyZNAuDLL7+kT58+1K1bN2dqCbTbeb29vfH19cXQ0JBOnTrh6+uLvr4+fn5+/Pbbi089KtLZRYw6MZG45SuIXbwYdVwcJvXqYj9qNGbNmub5xJ+SkMHiKbtJzEojxWMJK6JHMbWfP139KhaD9XmjTk4m/KPJJO/ejWWXLjh9NUVZkH6EIX8e5+rdRA591AZjw8fnvCNuJ7Dux9O06F8VU7sDdD/zPWPLN2VMpzlPbFORzlbID4p0dgklKyaGyN+mcbNNW6KmTcPYxxv3pX/jsXQp5s2b5ekQslRq1s84TGa6IMxjPhtjhzN3SN0S5RAA9M3NcZk5A/vx75C4ZQtBAweRGVp24hAXBuNaVyY6OZPVp0LyzK/gaUV5dwsu7gvFq1Z/WqgNWBlxmPTMZx/yUlAoTBSnUASkXbpMYIeOxMybh1nTpniuX4fbvHmY1svTUQPahaldSy6SGAanPf/mVmJn5r7enLY1HIvQ8vwj9PRwGDsWlz9mowoNJahXL5IPHy5us0oMDT1tqeNmzdz9t1A9ErLzPj6tXYiLSCX0ejzDq/YnVg+2HP+liC1VeNlRnIKOyQwJIWTUKPQtLfH6ZzMu06dhXLPmM+ud3XmHwJOxnHbZhmWWOd+8MZgmlUp+HASLVq3wXLMag/IOhIwYScyCBco6A9khO1tXJiw+jX/Oh+dZpnLd8hibG3Jxbyj1Gk2kpkqyOHADGpm3EwGU363CU3mez4fiFHRIVmwsIW+NgKwsXBfMp1zlyvmqF3QxmqPrbxJoexY7sxO8PexnarvZ6NjawsPIwwOPlSuxaN+eyF9+JWziJDQpihBfm+rlqV7Bgtn7AtFoHv9nNTDUp2azigRdiCYpQc1wl7YEiSwOXFiUZ3vGxsbExMQojkEhT6SUxMTEYGxsXKB6BTqnIISwAVyllBcKdJeXEE1qKiGjx6CKiMBt4ULKeeVvp1Ds3RS2zTtPtGkosc5L+bz7NrwqWunY2sJHz8wM59+mEuvjTeSvUwkKvInLrFkYubs/u3IZRQjBmFaVmLDyHDuv3qNDrQqPlfFu4czZ/4K5fCCcdh0+x2nFThZdWEArvzceK+vi4kJoaChRUVFFYb5CKcTY2BgXF5cC1XmmUxBC7AO6Zpc9DUQKIQ5LKSc9j5EvAzIri7BJ75F+6RIuM6ZjWufxLWp5kZ6sYu2006TJJI5VnstvLX7Cy7n0HgoTQmD35puUq16d8Envcbt3Hyr+/BMWrVoVt2nFRmcfJ37dcZ3Z+wJpX9PxsQ0GFrbGePo7cOVQOPVf82CIjS8/JV3m4u1d+Hi+kqusoaEhnp6eRWm+wktAfqaPrLL1iXoCS6SUDYFXnlHnpUVKScSUr0jet48Kn/8Pi1fy96tSqzUsn3aajMR0tldbwAeVWlHdu52OrS0azJs2xWPdWgxdXAgdM5ao2bORmifPk5dltCE7vTgfEs+RwLwP7fu0ciE9RcXNU5H0bD4FC42GxSd+LmJLFV5W8uMUDIQQTkBfoHAUl8ow0bNnE79mDXajR2GTrQmTH5bMPktaaCr7vFbymlUqbdp/q0Mrix4jFxc8li/DsstrRM+YSeg741EX4inM0kSvOi6Ut8g7ZCeAc1VrbJzMuLA3FFP7qvQp58zOtDBCo56uw6OgUBjkxyl8BfwH3JRSnhRCeAE3dGtW6SRuzRqiZ87Cqnt3HCZMyHe9PxddIPVyApcr7Mbc5gTjeq0CHUtXFAd6JiZU/PFHHD/5hOR9+wjq24+MEhCkpKgxNtTnreaeHL4Zw7mQ+MfyhRD4tnIm6k4S924nMrDRZPSApYe/KnJbFV4+nukUpJRrpJS+Usqx2e9vSSl76d600kXS3r1EfDkFs2bNcPr6q3zrEf2+6hIpx6KItLzKJdeN/NjyFwzMyp76632EENgOHYLbwr9QJyQQ1KcviTt3FrdZRc7Ahu5YmRgye2/eo4WqDStgZKzPxX2hOHq14VVhzvr4SySkKIvKCpCwaROq8Ly3Nr8oT3QKQoiZQogZT7p0Yk0pJe38ecImTsK4enVcpk/LpVv0JKSU/LzuEqn77pFpHMeWaov42qU9Fap0LAKLix+zBg3wXL8Oo8qVCXtnPJHTpr1UUtzm5QwY1sSDHVfucePe49NoRsYGVG/sxM3TkaQmZjLMbwxpQrBGGS281EgpiZo5i/CPJhOz4E+d3ONpI4VTaHcbPelSADKDgggZPQYDBwdc585B7ynSvffRaCRfrrtIyu4Iyumr2VD9d/obGNOy3a9FYHHJwbBCBdz/XoJV717EzJlLyJgxqBMSitusIuP1Jh6YGOrzx768p9B8WrmgUUuuHAqjqt9QmmbpsSx8P5lZj8dmUCj7yKws7v7vf0T//jtWPXrg+PFkndzniU5BSrn44QtY88j7l56s6GjujBgJgNv8eRjYP/vEcZZaw/trzpG47x72UrC3ylzcDCJ5t8cq0Hv5zhLqlSuH09dfU+HLL0k5eozbffqSfu3lUE63MTNiYEM3Np0PJyT2cY0ja0dT3GraculAOGqNZGjl3kQLydYTU4vBWoXiRJOaSsi4cSSsXYf92DE4ffdtvmYknodnfgsJIRoLIa6QHQtBCOEnhJitE2tKEZqUFEJGjSYrOhrXuXMw8vB4Zp2MLDXjlp8h6kgklbP0ue25hQjLAH5q8jWGliVL5K4oEUJg078f7ksWI9PSCOrfn8Rt24rbrCLhreae6AmYd+BWnvk+rVxIic/g9rloGjf5gKoqDYtvrFVOMb9EZMXEEDxsOCkHD1Hhyy9xGD9epzFU8vNoOg3oAMQASCnPAy10ZlEpQKpUhL47kfSAAJx/m4qJ77MjZKVmZvHW4lPcORtNowxD1O432VF+F1+Wb45rzZ5FYHXJx7R2bTzWrcW4Rg3CJr3HvZ9+RmZlFbdZOsXJyoRedVxYfSqEqKTHp4XcvO2wtDfm4r5QhKExw51aEEgmhy4VXnAZhZJLZnAwQQMGknHjBi6zZmLTv5/O75mv+Qop5aN6vy/PiuAjSCm5+/kXpBw8SIUvv8jX6dx0lZphf50gMCCG1zLKYVYxkz8rzKYv5nTo9LvujS5FGJYvj/uihdgMHEjsX39xZ8QIsuLiitssnTKqZSVUag1/Hb79WJ6ensC7hQvhN+KJDk2mY8svKZ+lZvH5J8dZUCgbpF24QNCAgWiSknBftBCLNm2K5L75cQohQogmgBRCGAoh3geu6tiuEkvU9OkkbNiA/dtvY9OnT77qfLXlCldvxTFEbY6ppT5LnaZQWZ3JB92Wg17eQcZfZoSRERU+/x9O331H2ukzBPXqTdpDwdnLGp72ZnTyceLvo8EkpD0esrNGUyf0DfW4uD8UQ3NHBlvV5Lg6gat3DhaDtQpFQdLevQQPG46emRkeK5Zj4u9fZPfOj1MYDYwDnIEwwD/7/UtH3IoVxMyZi3Wf3tiPG5uvOpvOhbHu6B1GCkv01ZKTnjNINEjk5/ofY2yj6NY8DeuePXBftgwpJcEDB5GwaVNxm6QzxrSsRHJGFsuP33ksz9jMkKoNHLl+PIL0FBW9m3+JmUbD4mPfF4OlCrombvVqQse9TTkvLzxWLM/XemVhkh+nIKSUg6SUjlLK8lLKwVLKvEVbyjBJu3YR8fU3mLdsSYUvvsjXQs+tqGS+X3ORNzNMMUzOIqvOYQ6Wu8ZnNnXx8htSBFaXfkx8vPFcuwYTPz/CP5pM5K9Ty+Qiq7ezFc2r2LPw8G0ysh6fnfVp5UJWpoaAo3excPSml5Ej21PvEBGb9+E3hdKHlJKoGTOJ+PwLzJo1xX3J4nztaCxs8uMUDgshdggh3hRCWOvaoJJI6pmzhL33Psbe3jj/NhVh8GzF8XSVmk//PE2vOEOshR5Ve6UyVy6nq7ocXV9bUARWlx0M7Oxw++tPrPv1I2b+fCK++qpMCuqNbOFFZFIGm84+flLVwdUCp8pWXNwfhtRIBjf8CIClh74sYisVdIFUqbj72WdEz56NVc+euP7+e77OPOmC/MhcVAU+A2oBZ4QQW4QQg3VuWQkh49YtQseMwaCCI65z/kDP1DRf9X768ywNg7KwMjeiwVvmfHvnE9yyNHza5W8wMNKx1WUPYWBAhS+/wO6tN4lfsZK7H39c5nYmNatsT00nS+YeyDsIj08rFxKj0rhzJRanyh3oIE1ZG3uepLSyvRBf1tGkpGjPIKxbj/3YsTh9+43OziDkh/zuPjqRHT+hARALvBSH11SRkdrIaQYGuC1YgIGtbb7qLV1yEdtzCehZG1GlfzRjTg1HqlVMrT0JU4caOra67CKEwOG993B4dwIJmzZrI7plZha3WYWGEIJRLb0IjEphd0DkY/le/g6YWhlxYW8oAMN83yJFwLrDXxe1qQqFRFZ0tPYMwqHDVPhqCg7j38nX1PSMvR9x/u4JndiUn8NrlkKIYUKIf4EjwF20zqFMo05O1h5Oi4/Hde5cjFxdn1lHo5FsWXSZhCNRRFnpY9PpMO9e+B/OWWpWNP6GKnXfKgLLyzZCCOxHj8bxk49J2rmT0LHj0KSlFbdZhUZnHyecrU2Yd+Bx6Qt9Az1qNXfmzuUY4iNTqVn7LRpmCf4O3YUqq+w4x5eFzKAg7RmEmzdx+X0WNn375qveviurmH9nG0cO6kZePz8jhfNodxx9JaWsKqX8SEpZprWPZGYmoe+8oz0wMn0aJt61nllHlaFm6+wLBB+7xwWzLDSNF/FT0DJaZAmWdF5BBeWAWqFiO3QoTt9+Q8qRI9x5a0SZic1goK/HW809ORkUx+ng2MfyazWviJ6e4NL+MNDTY5hXNyKFZPvpmcVgrcLzknb+vPYMQnIy7osXYdG6db7qJWYk8PWJ76mqUvNWu+k6sS0/TsFLSjkRrXPIN0IIYyHECSHEeSHEZSHElOz0g0KIc9lXuBBiY3Z6dSHEUSFERvZZiGJBajSEf/oZqUeP4fT115g3b/7MOikJGWz49QzBl2LYZZpCmv8PbEw4zevSgmkD9mDq5FcElr98WPfqhfOvv5B2/jx3hr9eZg659avvirWpIXP3Py59YWZVjkp1HLh65C6qDDXNmn5MpSwNiwNWlsldWWWRnDMI5ubaMwh++f9++HX7aGLI4nOv4RjaeOjEvvw4hUbPqX2UAbSRUvqhHWl0FEI0klI2l1L6Syn9gaPA+uzyscB44JcC9qFQiZo6lcR//sHh3Xex7tH9meVjwpNZ++MposOT2WIZQar3l1xURzLFpAqTBu9DvwzHRigJWHbqhMusmWTcuMGdoUNRRT4+F1/aMDUyYGgjd3ZevUdgVPJj+T6tXMhMy+L6iQiEkSnDyjfmGukcC1hbDNYqFIS4VdlnECpXLvAZhCM3NrM+/hLDNQ7s2VSfJfML9Jyeb3SmfSS13P9EG2ZfOY8yQghLoA2wMbt8pJTyJPD4kc4iInbJ38Qs+BPrAf2xGzXymeVDAmJZ//MZVJkattlfJrH696SQxjznzvTss07ZZVREWLRqheu8eajCwgkePITM0LDiNumFGdrEAyN9PRYcfHy0UKGSFfau5lzcF4qUks4tpmCnVrP4zKxisFQhP2jPIMwg4osvMGveDPfFiwp0BiElM5kpR77AQ5WFc+YUjDIksryxTmzVqfaREEJfCHEOiAR2SimPP5TdHdgtpUzMT1sPtTlSCHFKCHEqKqrwolAlbv+Pe99/j/krbanw2WfP3AFw9chdtsw4j6m1Ebsr7iTMYw62miyW1f6Q+u1+LJPhNEsyZo0aaqO5xccTPHgwGbce1xEqTdibl6N3XRfWnQ4jMik9V54QAp9WLsSEpXD3ZjxGVs4MMq/K4axYroUdLSaLFZ6EVKm4++lnRM/+A6vevZ7rDMK0HW9zV6r4uPxQIs5ncdsMBnSuohN7dap9JKVUZ08TuQANhBDeD2UPAFYU1GAp5TwpZT0pZT0Hh8KZmkk9eZLwDz/ExN8f519+Qeg/WY9ISsnxzbfYs+QqFatas995PpfsNuCfKVnW/k/caw8vFJsUCo6Jnx/ufy9BqlQEDxlCekBAcZv0Qoxo7oVKo2HR4aDH8qrWd6ScmUHO9tS+zT7HRKNhyZHvithKhaehSUkhZOw4Etavx37cOJy+/jpfh18f5tTtnayMOc0gjTnXLrYhU0rqdfXEyEA38VeeV/sof8I/2Ugp44G9QEcAIYQ92m2tWwvSji7IuHGDkHFvY+jsjMvs39EzfvKQTK3SsGvRFU5tC6JKQwc2237GUcMTvJpmwLy+27Fya1KElivkhXG1arj//TfC0JDgocNIO3euuE16bjzszejkXYG/jwWTnJH7oJ6BkT41m1Tk1rlokuPSsapYhx4GDmxLuc29+KDiMVghF1nR0QQPHUbKkSNU+PorHN55u8BxENJUqXxxcDIuqiw6u80kNSSFK/aCXs3cdWR1/k40Rz+qfQR88qx6QgiH+7IYQggToB3Zi9VAb2CLlDL9CdWLBFVEBHdGjESUM8J1/nwMbGyeWDY9RcXmGee4fvwetTraM09vJKc0wfRJsObrYQcwtH72OQaFoqGclycey5aib21N8BtvknLsWHGb9NyMbFGJpPQsVp54XCjPu6UzUkouH9TKYgyuPwkNsFyRvih2Mm7fJqj/ADJu3dKeQcinovKj/L57IndkJp9X7M6hf9O4p6+hY7cqGOrrLkrj87acn1MWTsBeIcQF4CTaNYUt2Xn9eWTqSAhRQQgRCkwCPhNChGYvRusEdWIiISNGoklKwm3ePIxcnJ9YNjE6jfU/nybidgI1uhvyVfxQQjSJdL5XhbeGbMfIxEJXZio8J4bOzrgv/Rsj54qEjBxF0r59xW3Sc+Hvak1DT1v+PHQblTq33pOlvQkePvZcPhiGWqXBtVoXXtEYsyb6NCkZBVqqUyhE0s6dI3jAQDSpqbgvWZyvmCt5ceHOQf6OOExfdTnUycPJSsnispM+Peo8+buqMHhep/DMMZCU8oKUsraU0ldK6S2l/OqhvFZSyu2PlI+QUrpIKS2llNbZr3XyydZkZhI67m0ygoJwmTUT4xpPlp6IuJ3A2h9PkZqYiXOXKD6KGIu+OgPf4Ja80m0OFW2KR7RK4dkYli+P25IllKtaldC33ym1IT5Ht6zE3YR0/jn/uFCeTytn0pJU3DwTCUIwrNZQkgSsV9YWioWkPXsJHv46epaW2jMIPj7P1U6mOpPP979HebWG4f6zOL83jAtGWQzpXBUDHY4S4ClOQQhh+4TLjnw4hZJMwqZNpJ48ScXvvsWsceMnlrt1NoqNU89iaKSPpvUhPo34mioqNWa3B+HV8B3aVHcsQqsVngcDGxvcFi3ExN+PsPfeJ35t6dvL36qaA9UcLZh34NZjB9Rcq9ti7WjKxX3aBWffemOpo4KlwdvJ0pQtwcCSTtzKVYS+/TblqlTRnkFwf/55/7l7PiBQk8bnTu04sduQDD0IcTGki5/uY7k/zeWcBk5l/3z4OgWUaqEV6969cV++HKsuXfLMl1Jybtcd/p13ETtnM677zWVm7Ao6ZOoRHfoe+s5teL99tSK2WuF50Tc3x23+fMyaNuXuZ/8jdsmS4japQAghGNHCi4CIJPZfz70NW+gJfFo5c+92IpHBiaBvwHCPzoQLNbtO/1FMFr9cSI2GyF9+IeLLLx+cQbCze+72roaf4M+w3XTNMsTB8n0iAhPYa5TJ2A7V0NfT/fP4E52ClNJTSumV/fPRy0vnlukQIQSmdWrnmafRSA6uusHhtTdx9bZkq/NkNqadYQy23Er5gUh9V2YOqK3ThR6FwkfPxASX2b9j0b499777nug//ihVshBd/SpSwdI4T+mL6o2cMCynz8Xs7aktm32CR5aahVf/LlV9LI1oUlIIfWe89tBr/37aMwj5lNfPC5VGxed7xmOj1jCh6TQObwwixhgyXI3p7ONUiJY/GeWb7SEy07P4948LXNwXikcTE2abjeJSVgQ/W/gRbjGNMxEapvb1o6K1SXGbqvAc6BkZ4Tz1V6y6dSNq+gwif/ml1HxpGhno8WYzT47eiuF8SHzuPBMDqjWqwI1TkaQlZaJnbMlQ+wZckWmcuvFP8Rj8EqC6e5egwUNI3rsXx08/1UZkLOAZhEdZuP8zAtQpfFa+KVfPOJKeomKLQToT21VDrwhGCaA4hRxSEjLYOPUswZdicGmTzjeakaRr0vjLqz8ZXj+y9EQ4o1p6KesIpRxhYIDT999hM3AAsX/+RcSUKaUmilv/Bq5YGBsw78DjowWfli6oszRcOaxdjO7Scgq2ajWLT00rYitfDtIuXuR2376o7tzBde4cbIcMLvAZhEe5GXmeOcFb6ajSw6fm91w6GMYNS7BztaBDrQqFZPmzUZwCEBOmFbWLu5eKWZvrfJH6IRWz1KxoOAXzmhP5ZP1F6rrbKOsIZQShp4fj//6H3Yi3iF+5ivDJk0tFFDcLY0MGNXTn30t3CY5JyZVnW9EM52o2XDoQhkatwdjanf6mXuxXRXEr4kwxWVw2Sfz3X4IHD0GvnDEeK1fkS0n5Wag1ar7YOQ5zjYaPWvzK/tW30TPWZxtpvPtKlSIbJYDiFAi5Gsv6n0+jUUviGv7DL6m/00wlWPLqMmyq9mTc8rMYGugp6whlDCEE5d97D4eJE0nc/A+h775bKqK4vd7UAwM9PRYcfFzbybeVC8mxGQRdjAGgX9P/UU6jYYkSma1QkFIS9fvvhE2chHGtWnisXkW5KoWjP7T08DdcyErgY9t6RETUIDI4icMWaqo4W9K+ZtHOTjxtS6qPEOKYECJECDFPCGHzUJ5u4sAVMVePhLNl5nlMbYw4U3May1T/MUxjxvQBuzCrWJuvtlzh6t1EZR2hDGM/aiSOn35K8q7dhI4egyY1tbhNeiqOlsb0qO3M6lMhxCRn5Mrz8LXD3LZczvZUW9eGdNO3YXPSDaITQ4vD3DKDJiOD8Pc/IHrmLKy6dcNt0cJ8h+d9FsExAcwMXEtrFbRoNp1jGwMxdDLhUEYak9pVfeFpqYLytEffP4AvAR/gOnBICFEpO6/4okoXAg9E7QKw8yrHBrcPOSQD+MK4Eu8P2Y++uSObzoWx/PgdZR3hJcB2yGCcvv2WlGPHSkUUtxEtvMjI0rD4aHCudD19PbxbOBMaEEdsuHZ6aUid8WQBKw5NKQZLywZZUVEEDx1K4tatOEyahNMP36NnVDiy+Bqp4Yv/RmOk0fBZs285tlUbPGmDSMPP1Yo21csXyn0KwtOcgoWUcruUMl5K+QvwNrBdCNGIh+IilEauHArn1LYgyvvC77ZjuStjmFuxI737bgCDctyKSlbWEV4yrHv1xHnqr6RdvMidYcNLdBS3yuXNeaWGI38fDSI1M/daSM2mFdE30OPifu3IwKNWH1prjFgVeYzUjMcD9ig8nfRr17jdrx8Z12/gPGM69iNHFOqT++pjP3NaFcMHlt5kGbQg4GgEhjWtuJqaxsRiGCXAM9YUhBBW919LKfcCvYC/Ad1J9BUB1Rs7Ydf6Lt+bTMBCk8Ey//do0P4XEIJ0lVpZR3hJsezYEddZM8kIDCR4yBBU90puFLfRLb2IS1Wx5lTuaSETCyOq1CtPwLEIMtKyQAiGVx9EgoBNx38qJmtLJ0l79hI8YCCoNbgv/RvL9u0Ltf2w+NtMvbaUJpmSrh3ncmDFNcxtyrE4Po46bta0rFo8URuf9o33I5BLFEhKeQFoy4MQmqWSbad+5dv0H6idpWHZK3PxqPNmTp6yjvByY96yJa7z5pEVfpfgwYPJDC2Zc/H1PGyp627D/IO3yHpEKM+ntQtZGWquHbsLgH+Dd/BVSZbc+ge1Jl/xsV5qpJTELFxE6LhxGHl64rF6NSa1ahX6PaZsH4GQGr5o9DkXjyUSE5aC2t+akKR0JrWrViyjBHj6ieblUspjAEIIcyGEeXb6HSnliKIyUBe0cnuFsVgzp/c2rDweRBZV1hEUAMwaNsBt0ULUiYkE9R9A6pmzxW1Snoxq4UVoXBrbLkXkSi/vbomjpyUX94UhNRJhYMRw13aEiiz2nFtQTNaWDmRmJhGff07kjz9i0b497kv/xtCx8Of1N56eydGMe0wyrYKlSzdO/HMbl5q2zL15lwYetjSt/PwyGS/Ks6aPxggh7gDBwB0hRLAQokABdkoiFhVrM2bYQQxtHsyCKesICg9j4uuLx/Jl6JmZEjxsWIkU0nulhiNeDmbMOxD42Mlsn1YuxN9LJTRAuzbSpvnnuGSpWXR5Yak5xV3UZMXFceetEcSvWYvdmNE4/zYVPZPCny2ITArn50vzqZepoU+XPzm87gYajSS6qin3kjJ4t12VYhslwNO3pH4GdAFaSSntpJS2QGugU3ZemUFZR1DIi3KVKuG5ejVmDRpw97P/EfHNt0iVqrjNykFPTzCyuReXwhI5EhiTK69ynfKYWBhyIXt7qr6pDUNt/bmgSeFc4Pa8mnupybh1m6D+/Uk7e5aKP/1I+QkTEHqF/z0gpeTr7W+ikhqm1PuAsGDBzVOR+L7iypzTd2jkZUuTSvaFft+C8LReDwF6SilzztRnv+4LDNW1YUWJso6g8CT0ray0Mgavv07c0qXceWtEidqZ1L22Mw4W5ZizPzBXur6hHrWaOxN0MZrE6DQAurX4Ciu1hkWnfi0OU0ssKUeOENS/P5qkZNwWL8aqa1ed3evf8wvYlxrKO+XccPYeyoGV17FyMOGylSQqKYOJr1TV2b3zy9OcgswrXKaUMg0oHWIx+UBZR1B4FsLAAMePPsTph+9JO3uWoD59Sb92vbjNAsDYUJ/hTTw4eCOay+EJufJqNa+IEIJL+8MAMLWrTD8TV/amRxAcebE4zC1xxK1cyZ0RIzF0dMRj9eonqicXBjEpUXx/bia+mWoGdV3E2V13iL+XSsPelZhz8DbNKtvT0Kv41hLu8zSnECaEaPtoohCiDXBXdyYVHco6gkJBsO7eHfelfyMzMggaMIDEHTuK2yQABjd0x8xIn/mPCOWZ2xjj5W/PlcPhqDK1u44GNP4UA+Dvw1/l0dLLg8zKIuK774j4cgpmzZrivmL5U0PyFgbfb3+LFDR85fcOKemWnN4WRKXaDuyOTyImJZOJ7QpHMuNFeZpTGA/MFUIsEkK8k30tBuahPchWqlHWERSeBxNfXzzWrqVclcqEjZ9A1MxZxa6yamVqyIAGbvxz4S6hcbllOnxauZCRmsWNk/cAsPdoThdhycaEq8Qm3ysOc4sddXIyIWPHErfkb2yHDcV19mz0zc11es9dl5byX/ItxhhUoFL90RxcfQP0BLW7eTJ3fyAtqzpQ171wZDNelKdtSb0MeAMHAI/s6wDgnZ1XqlHWERSeF0PH8rgvWYJVjx5E//47YRMmoE5OeXZFHfJGM08E8Oeh3EJ5FatYY1vRjIv7QnN2HQ31H0uGEKx6CaUvMkNDCR4wgJQjR6kwZQqOH3+M0NfX6T0T0uL45tTP1FCpGf7aQm5fjCHoQjT1O3uw9moEcakqJrYr/rWE+zxt91FloK6U8i8p5XvZ159A3Yc0kEolWy/cVdYRFF4IvXLlcPruWxw/+Tj75OsAMkNCis2eitYmdPWryMoTIcSnPlB7FULg08qF6JBkIm4lAlDJdxAtsvRZGXGIdFVacZlc5KSeOUNQn76o7kXitmA+Nv36Fsl9f/pvJAmo+armW2BakYOrrmPjZEalphWYd+AWbauXx9/VukhsyQ9PmzOZBiTmkZ6YnVdqaeBpy4jmnso6gsILIYTAduhQ3ObPQxUZSVDvPqQcPVps9oxs6UWaSs3SY7mF8qo2cMTIxCBHPRUhGF61L7FCsu7od8VgadGTsGkTd4YNR9/SEo9VKzFr1KhI7nsgYC2bEwJ4U9hRvfFEzmwPJikmnZYDqrL42B0S0lS8WwJ2HD3M05yCo5TysS0K2WkeOrOoCHCwKMennWsq6wgKhYJZkyZ4rlmNQXkH7rw1gtglxRMbuXoFS1pVc2DRkSDSVQ/kLIyMDajRxInA05GkJGjltus1eo8GKph7axPJ6Xk9+5UNpEZD5NTfCP9oMiZ16uCxaiXlPD2L5N7JGUl8dfwbKquyGPnaQuIj0zizI5hqDStg7mrO/IO3aFfTER8Xq2c3VoQ87VvR+il5z5yEF0IYCyFOCCHOCyEuCyGmZKcfFEKcy77ChRAbs9OFEGKGEOKmEOKCEKJOQTqioFCcGLm54b5iJeatWnHvu++4++lnxRK0Z1SLSkQnZ7LuTG7NJu+Wzmg0kssHteE6hWE5JnmPIE5I/tr3YZHbWRRoUlMJm/AuMfPmYd2nD24L5qNvbV1k95+6YwxRMouvqg7G0NaT/SuuYWCoT5Nelfnz0G2S0rN495WSsePoYZ7mFE4JIR7TOBJCvAWczkfbGUAbKaUf4A90FEI0klI2l1L6Syn9gaM8ENfrBFTJvkaijeegoFBq0Dc3w2XmDOzHjiVh/XruDBmKKrJolVYbedni62LF/AO3UGsejFasy5viVsuOywfCUGdpd0vVavA2ndRG/B1xiHsJd4rUTl2jCg8nePAQknbtovzkj6jw1RSEYdGFgTkeuI01secZihU+zT7m5ulIQgPiaNjVi0wD+OvQbTp5V6BWxZI1SoCnO4V3gdeFEPuEEL9mX/uBN4EJz2pYarkv4G6YfeV8SoUQlkAbYGN2UjdgSXa9Y4C1EMKpoB1SUChOhJ4eDuPfwXn6dNKvXyeodx/SLhbdQTEhBKNaVCIoJpWdV3IL5fm0ciY1MZNb56K0CXp6vNPwE7KAP/ZMKjIbdYnMzCRmwQICX+tCZlAQLrN/x2748CLVEkrNTOGLQ5/hrspiXOcFZGZqOLzmBvau5ni3dGb+wVukZGaVuLWE+zxtS+o9KWUTYAoQlH1NkVI2llJGPKnewwgh9IUQ54BIYKeU8vhD2d2B3VLK+xOazsDD2zdCs9MebXOkEOKUEOJUVFRUfsxQUChyLDu0x2PlCoShIcGDBpOwaVOR3bujdwXcbE35Y/+tXGsb7rXssLQ3frDgDLjW6kV/PVs2JAQQGHGmyGzUBSlHjnCrew8if/kVs8aN8dy8CYvWrYvcjpm7JhCGiimevTB2qMGJLbdJScyk5cBqxKepWHg4iM4+TlSrYFHktuWHZ660Sin3SilnZl97CtK4lFKdPU3kAjQQQng/lD0AWFEga7VtzpNS1pNS1nNwKJ4gFAoK+cG4WjU81q7BxN+f8I8mc+/Hn5BZWc+u+ILo6wlGtPDifEg8J27H5qQLPe321Ls3E4gKeRBydGSrHzCVkmn7PtK5bbpAdfcuoe9O5M4bbyKzsnCdOwfX32dh5OJS5LacDdrDsshjDFCbULf1FGLCkrmwJ5SazSpSwdOKuQcCSVOpS+Rawn2KZPuNlDIe2At0BBBC2AMNgK0PFQsDXB9675KdpqBQajGwscHtzwXYDBpE7MKFhIwajToh4dkVX5A+dV2wNTNi3iPSF9UbO2FgpJdrtGDj1oQ3TbzYlxHBqZvbdG5bYSEzM4meP5/AVzuTvG8fDhPG4/XPZsxbtiwWe9Kz0vn8wIc4qTW822keUuixf8U1ypkY0LhbJaKTM1hyJJhufhWpXL5kjhJAh05BCOEghLDOfm0CtAMCsrN7A1seEdzbDAzN3oXUCEiQUpYJjSWFlxthaEiF/31Gha+/IuXECW737UtGYOCzK74Axob6DGvswe6ASK7fezAqMDYzpGrDClw/cY/0lAcy4IPbT6N8lpqpR78uFfEWkg8f5la37kT9OhWzpk3w2rIF+zFj0CtXrths+mP3JIJkBl+6dMLUyZ9rxyK4ezOBxj0rYWxuyNz9gWRkqRnftuSOEkC3IwUnYK8Q4gJwEu2awpbsvP48PnW0DbgF3ATmA6U+mI+CwsPY9OmD++JFaJJTCOrbj6Q9e3V6v6GN3TEx1H9stODT0gW1SsPVww+euYxtK/G2fX0uapLZUYKjs6nCwwkdP4GQN99CatS4zp+H66xZOhezexaXQg+z6O4BemUZ0fiVH0hPUXFk/U0qeFlSo7ETkYnpLDkaTPfazng56FZn6UXRmVOQUl6QUtaWUvpKKb2llF89lNdKSrn9kfJSSjlOSllJSukjpTylK9sUFIoL0zp18Fy7BiMPD0LHjSN6zlydPZnbmBnRr74rm86FEZHwYFBu72JOxSrWXDoQiuahbatdX5lKZZWaGRf+QKUuOcGEADSZmUTPnUdg59dIPnAAh3cn4PXPP5g3b17cppGamcLkPRMor9Ywqf3voG/I8U23SE9W0WJANYSe4I/9gWRpJOPblOxRAhTRmoKCgsIDDJ2ccF+2FMvOnYmaNo2wSZPQpKY+u+Jz8GYzT9QaycLDuYXyfFq5kBidTtCF6Jw0fTM7Jrq/xh1UrDnyrU7seR6SDx7idpeuRP32G+bNmlFp6xbsR49Gz8iouE0DtNpGdzTpfOfWBUvXRkQGJ3LpYBg+rV1wcLUgIiGdZcfv0KuOMx72ZsVt7jNRnIKCQjGgZ2xMxZ9/ovwH75O0/T+CBg1GFVb4+ypcbU3p7FuRZcfvkJj+4Onf098eS3tjTm0LyjVSad7qK+qrYG7g+mKXv1CFhxP6znhCRmjP0LrOn4/LzBkYOhfvVNHD7L68jHWxF3gTK+q3/Q6NRrJ/+TVMLYxo0MULgNn7bqLRSN4pBaMEUJyCgkKxIYTA7s03cZ07B1VoKLd69uLe9z8UelS3US28SM7IYvnxB6eW9fX1qNvJg6g7SQRffBDfWRgaM6nWm8QKycJi2qKqycwkes5c7a6igwdxmDgRz382Y968WbHY8yQik8L54uSP1MzMYmy3paCnz5VD4UQGJ9G0T2XKmRgQFp/GyhMh9KnniqutaXGbnC8Up6CgUMyYt2iBx6pVmDVsSOzy5dzu1o3bvXoTu3QZ6vj4F27f29mKZpXt+evQbTKyHgjlVWtUAUt7Y05suZ1rtODdcDyd1IYsiThEZELRyoEnHzyonSqaNg3zFi2otG0r9qNGlpipovtopIbPtg0jU6r5ofZEDG08SU3M5NjGQJyrWVOlnlaS//e9N5FI3m5TuZgtzj+KU1BQKAGU8/LEZcZ0qhzYj+MnnyA1Gu598w03mrcg9N2JJB84gFSrn93QExjZwovIpAw2nQvPScs1Wrj0YLTwQP5CMruI5C8yQ8MIefttQkaMBCFwXbAAlxnTMaxYsUjuX1CWHfmOo+kRfGBaFc96IwE4uuEmqgw1LfpXQwhBSGwqq0+G0L++G86lKJCX4hQUFEoQBjY22A4dgteG9XhuWI/1gP6kHjtGyMhR3GzdhshffyXj1u1nN/QIzavYU8PJknkHbuXacXR/tHDykdGCa63e9Bc2bEi4SmDE2ULpW15oMjKI/uMPbr32GimHj+AwaRKemzdh3qypzu75olyLOMNvN1bROhN6d10EQPiNeAKORuD/ihu2TtrF5N/33kRPTzC2demKSaY4BQWFEopxjRpU+OQTqhzYj/OM6RjXrEnMXwu59eqrBPUfQNzq1aiTk5/dENr1i9EtvbgZmcyegAfKrfdHC5HBj4wWgJGts+Uv9utGWjv5wAFude1K1PQZmLdsqd1VNHJEiZsqepj0rHQm7xyDlVrNl61/Q5hYoVZp2LcsAAs7Y+q96gFAcEwKa06HMrCBG05WpWeUAIpTUFAo8QgjIyzbt8d1zh9U2beX8h98gDo5iYjPv+BGs+aEffAhKUePIjWap7bzqo8TztYmzD2Q+zR1tUYVsLB7fLRg49aUN0082ZcewenAfwutP5mhYYSMe5uQkaMQevq4/rkAl+nTSuxU0cNM2/k2NzWpfOPUFtvKrwBwansQcRGptBpYDcNy2njPM/fcxEBPMKZV6RolgOIUFBRKFQYODti9+QZe//yDx+pVWPXoTvL+/dx5/Q0CX2lH1IyZT4wVbaivx5vNPDkZFMfp4LicdH19Peo9YbQw6JXfCk3+QpORQdTs2dzq3JmUo0dxeG8SXps2Yt605E4VPczh65tYFnmcQWoTmnb4DYCY8GTObA+magNH3GrZAXA7OoX1Z0IZ3MgdR0vj4jT5uRClQefkSdSrV0+eOqUcfFZ4udFkZJC0axcJ6zeQcuQISIlp/fpY9eyJZYf26Jk+2AqZkpFFkx/20MjLlrlD6uWkq7M0LPviGCbmhvSeXC9X/IENG4fyecJZfvWbQHv/t55uS2YmqrAwVKFhqEJDyAwJRRUSQmZYKKrgO2hSUrDo1BHHjz7CsEKFwv9l6IjY1Ch6rX4F66xMVnbbQDmH6kiNZP0vZ4i/l8rALxtiYqGd9pq46hz/XrrLwQ/b4GBRfFpMT0MIcVpKWS+vPIOiNkZBQaFw0StXDqvOnbHq3BnV3bskbNpM/Ib13P34Y+59/TUWnTpi3aMHJnXrYlbOgKGN3Zm19yaBUclUytbh0TfQjhb2Lg3gzuVY3L3tctrv2m4qS1a0YPr52bTyGYpebAKZoaGoQkPJDAlBFZL9OjSUrIgIeOhBUxgZYejigqGLM6b+/li0b49Zo0ZF/jt6EaSUfLF1OAmomVNzJOUcqgNw6UAYEbcSaDu8Ro5DuBmZxKZzYYxo7lViHcKzUJyCgkIZwtDJCfvRo7AbNZK0M2eIX7+epH+3k7BuPYbublj36MGgtp2Ye0CPBQdv8X1P35y61RpV4NS2IE5sDsTRKCbX0/4XZ8tzL+Ie13+tg35m7q2xBuXLY+jiglmD+hi6uGLo6oKRiwuGrq4YODgg9Er3LPXak7+xL/UOHxq5Uq3xuwAkx6VzdGMgrjVsqNbwwYhn+u6bGBvqM7KFVzFZ++IoTkFBoQwihMC0bl1M69ZF8+mnJO7YQcL6DURNmw7TZ/BHZV9WB9XkTtpVjKMjtA4gJASXFEeuOnfl1PAfsYu9AoCeqSmWLs7csY5kr6ekT9v3MfesrB0BODujZ1z65s3zy+3oq/x0ZSGNVRoG9f4bhEBKyYGV15FqScuB1XOm2q5FJLHlQjhjWlbCzrx0jhJAcQoKCmUePVNTrLt3x7p7dzJDQkjYsBHWb+DdG+dJOQUpenoYOjlh6OJC1UoWBCeqCGsxhjr97TFyc0PfxgYhBKlHf+PT638hKpzm7ZZvFne3dI5KrWLy9jcx1qj5ptn36JnZA3DrbBS3z0fTuGclrBwebDedvvs6ZkYGjGheekcJoDgFBYWXCiNXVxzGv4P92+P49Y8tbLwWz6KPu1LZyTqnTIODYexbdo0oQ1fcbW1z0r0bTqDjlb9Zcvcg/ZLCcLAoOcJ0umD2nve4ok5imn1jytfoBkBGqooDK69j72qOf9sHgSIvhSWw7WIE77SpjI1ZyT1nkR9K92SfgoLCcyH09Bg+rAPxNo78sONmrrzqjZ0wty332LkF9PQY33AyKiSzd08sYouLlpO3dvBn2B56ZRnS9tXZOelHNgSSlqyizZAa6Olrvz6z1Bomr7+AvbkRbzUr3aMEUJyCgsJLi715Oca0qsSuq/c4GvjgfML9nUj3bicSciU2Vx1X7770E9asj7/CrXvnitjioiExI4FPDn6EW5aaD1/9Cwy06wPhN+K4cjAcv7auOLg9iLE8/+BtLoUl8lU3b6xMDYvL7EJDcQoKCi8xbzbzpKKVMd9uu5JLE+n+aOFRBVWAka1+wERKpu3TjfxFcSKl5JutrxMtVfxQZRCmTv4AZKnU7F16DUt7Yxp08cwpHxiVzG+7rtOxVgVe9XEqJqsLF8UpKCi8xBgb6vNBx2pcCktk47kHQX70DfSo2zHv0YKtezPeNPZgb/pdztza/miTpZot5+bxb9INxuqXx7vFpznpp/8NJv5eKq0GVsfQSCtlodFIJq+7gLGBHl91q1VcJhc6ilNQUHjJ6ebnjK+LFT//d420h84g1GjihLlNOU5ufXy0MLidVv7i1yNf6SzGdFETGn+L787Pok6mmje6L4PsraYxYcmc+S+Yag0r4FrzwcL70uPBnAyK43+v1aR8KZSzeBKKU1BQeMnR0xN8+moN7iak8+ehWznp+gZaBdWIW4mEXM09WjCxr8o4u7pcUCex6/xfRW1yoZOlyeKTba+D1PB9o8/Rt9BOBWk0kr1LAzAyMaBpnweBckLjUvnx3wCaV7Gnd12X4jJbJyhOQUFBgYZedrSv6cgf+wKJTErPSc8ZLeSxttC13VQqq9RMP/87Ko3q0SZLFX/u/5Szqlg+s/Sjok//nPRL+8O4dzuRZn2qYGKu3WoqpeSTDZeQwHc9fHLpRJUFFKegoKAAwORO1cnI0vDbzhs5aQ+PFkKvxuUqb2DmwLtunQhGxbojPxS1uYXGhZBD/BG8lVdV+nTu+mdOelJsOsc2BuJW05aqDRxz0tedCePA9Sg+6li91MRdLgiKU1BQUADAy8GcwY3cWXXyDtciknLSazTWjhby2onUotU31FNJ/ri5hpSMpEebLPGkZqbw8d4JOKrVfNr+DzDUnlDOkbKQkpYDq+WMBiKT0vl6yxXqudswpJF7cZquM3TmFIQQxkKIE0KI80KIy0KIKdnpQgjxrRDiuhDiqhBifHa6jRBigxDiQnY9b13ZpqCgkDcT2lbBvJwB3227mpOmb6hH3Y7uRNxKeGy0IIxMmFTzdWKFZNG+yUVt7gvz4/YRhGoy+M69O5ZujXPSA89EEXQhmgZdvLC0fyBl8cWmy6Sp1PzY2xc9vbI1bXQfXY4UMoA2Uko/wB/oKIRoBAwHXIHqUsoawMrs8p8A56SUvsBQYLoObVNQUMgDGzMj3mlThf3XozhwPSonvUaTik/cieTTaCIdsgxZfPcAUUlhjzZZYtl1aSnr4y7yprCmbptvc9LTU1QcWHUdBzcL/No8WET+9+Jd/r0UwbuvVMmRHC+L6MwpSC33A8gaZl8SGAN8JaXUZJe7HzC2JrAnOy0A8BBCOKKgoFCkDG3ijputKd9tu4o6+0Db/dHC3cAEQgNyjxbQ02N8gw9RIflj96RisLjg3EsK48tTP+GdqWZM12XwkLz30fU3SU9W0Xpw9Rwpi/jUTP636TK1KlqWesG7Z6HTNQUhhL4Q4hwQCeyUUh4HKgH9hBCnhBD/CiGqZBc/D/TMrtcAcAce2+slhBiZXfdUVFTUo9kKCgovSDkDfT7qWJ2AiCTWnn4Q2jNntJDH2oKbT3/6CmvWx1/m1r3zRW1ygdBIDZ9uHUamVPN9nUkY2jxYGwi7HseVw3fxf0TK4ustV4lLzeSn3r4Y6pftpVid9k5KqZZS+qP9cm+QvU5QDkjPDgU3H7i/yfkHwDrbibwDnAXUebQ5T0pZT0pZz8HBQZfmKyi8tLzqU4E6btb8suM6KRlZgHa0UKdD9mjhWtxjdUa1+h5jKZlewuUv/j78Dccz7vGRWTU86j4IL5qlUrNvmVbKov5DUhb7rkWy7kwoo1t6UauiVXGYXKQUicuTUsYDe4GOQCiwPjtrA+CbXSZRSvl6thMZCjgAtx5rTEFBQecIIfi0c02ikjKYe+DBv2HNphUxs857tGDr3pw3jN3Zkx7O2Vs7itrkfHHt7imm31xDm0zo2XVRrrxT24Iek7JIzsji0w2XqORgxjttquTRYtlDl7uPHIQQ1tmvTYB2QACwEWidXawlcD27jLUQ4r4Q+VvAASlloq7sU1BQeDp13W3o7OvEvAOBRCRoD7TlrC3czHu0MOSV+/IXU0qc/EV6Vjof7RqLtVrNl22mI4wtc/JiwpI5+98dqjXKLWXx0/YAwhPS+Km3L8aG+sVhdpGjy5GCE7BXCHEBOIl2TWEL2mmiXkKIi8D3aB0AQA3gkhDiGtAJmKBD2xQUFPLB5I7V0Wjglx3XctJqNHV64mjBxKEaY23rcF6dyO4Li4rY2qfz245xBGrS+KZie2wqtclJz5GyMDWgae8HUhYnbsey5Ggwwxp7UNfdNq8myyS63H10QUpZW0rpK6X0llJ+lZ0eL6XsLKX0kVI2llKez04/KqWsKqWsJqXsKaV8/DFEQUGhSHG1NWV4Uw/WnQnlcngCAAaG+jmjhbA8Rgvd2k2lkkrN9HOzSoz8xcFr61kedYLBGjOatP8lV96l/aGPSVmkq9R8tO4CLjYmfNChWnGYXGyU7WV0BQWFF2Zc68pYmxjy7darOSODGk2dMLMyyvOUs4F5ed5160gQmaw/+mNxmJyLmJQo/nd0ClVUWbz72mLQfxCFWCtlcesxKYtpu25wOzqFH3r6Ylbu5YparDgFBQWFp2JlYsiEtlU4EhjDngDtsSIDQ33qdPTQjhauxz9Wp2Wrb6mrksy+sbpY5S+klHyxbShJUs2PtUZTzqFarrwDK649JmVxMTSB+Qdv0beeC82q2BeX6cWG4hQUFBSeyaBG7njZm/HdtqtkqTUA1GymHS3ktbYgjEx4L1v+YvH+j4vDZADWnPiV/amhTCrnRpVG43Pl3TwdSdDFGBp2fSBloVJr+HDdBezMjPi0c83iMLnYUZyCgoLCMzHU12Nyp+oERqWw4qT2QJt2tOBO+I34PEcLPo3epX2WAYvC9xOdFF7EFsOt6Mv8fHUxTTMlA7svzQmaA1opi4Orb+DgZoFv6wdnZOfsC+Tq3US+7u6NlUnpj7f8PChOQUFBIV+0q+lIQ09bpu28TlK6dgG5ZrOKmGaPFh5DT58JxSR/oVKrmLx9BCYaNV83/wFhmnv30JH7UhZDHkhZ3LiXxMw9N+ns60SHWhWK1N6SxMu1gqKgoPDcCCH4rHNNusw6xOx9gXzUsXrOTqSDq24Qdi0O52o2ueq4efenz5lZrI6/xODIi3iW93khG6SUpKhSiE6LJiotipjku0TFBxGdFEp0SgTR6TFEZyYSmZVMvMxihkNTHKq/lquNsGtxXD18l9rt3XBw1UpZqDWSD9ddwLScPlO6lp14y8+D4hQUFBTyjY+LFT1rO/PnodsMauiGi40pNZtV5PT2YE5suU2PR5wCQjCq5Xds3juW6Xs/YFq/7Xm2m6XJIi49jqi0KKJTo4hOCCI6MYSopDBi0qKIzognSpVEjDqdNDSP1TeQEnu1GocsNRXVGvz0DKlj6UXrTr/nvk+mmr3LArRSFq89kLJYfCSIs3fi+a2fH/bm5V78F1WKUZyCgoJCgXi/QzW2XrzLz/9dY3r/2tq1hQ7uHFqd92jBzqMlrxu7MSs9lN8PfIZKnUl0agTRadqn+uisFGKlirzOP1uoNdove7UaHylw0DfG3tAS+3I22Js64GBeEXtLd6ys3BEWjmDuCGYOubadPsypbUEkRKbRdYJ/jpTFnZhUfv7vGq2rOdDd37mwf12lDsUpKCgoFIiK1ia81dyT3/cG8npTT/xdranVrCJn/gvm5NbbjzkFgCGvTGPt5h7Mub0JAymxzf6ir6DW4C2MsDc0w8HICntjO+zNKmBv4Yq9tTvlLF3AvLz2y77ci8UwiA5N5uyOO1RvVAHXGto1BiklH2+4gL6e4NsyGG/5eVCcgoKCQoEZ06oyq06G8O3WK6we1RgDI33qtHfn0JobhF2Pw7lqbsdg6lCNjS2mkZFwB2srd/QsnLRf9Ka2oKd7TaH7UhblzAxo2vuBsN3qUyEcvhnDN929qWht8pQWXh6U3UcKCgoFxrycARPbVeVkUBz/XY4AoFbziphaGnFyax47kQCzKu2xrfcWelXaQQVvMHcoEocAcHFfKJFBWikLY3PtVtN7iel8s/UqDT1tGdjArUjsKA0oTkFBQeG56FfPlSrlzfnh3wAyszTa0UIHd8KuxRN2veRIlyXFpnNs0y3catlSpb5WykJKyacbLpGZpeGHXmU33vLzoDgFBQWF58JAX49POtcgKCaVpceCgWePFooaKSX7l18DKWk54IGUxZYLd9l19R7vta+Kp71ZMVtZslCcgoKCwnPTqqoDzavYM2PPDRJSVblGC+E3in+0cPNUJMGXcktZxKZk8uXmy/i5WPFGU89ntPDyoTgFBQWF50YIwSev1iAhTcXMPTeAB6OFE1uCitU2rZTFdcq7W+DbxjUn/at/LpOYruLH3r4YlPF4y8+D8htRUFB4IWo4WdK3riuLjwYRHJOCgZE+tdu7EXYtjvAb8cVm1+F1N0lPydJKWWSvGewJuMfGc+GMbVWZ6hUsn9HCy4niFBQUFF6Y99pXxVBfjx+3BwBQq4UzJsW0tpAYnca5XXcIOHKX2u1csXfRSlkkpqv4ZP0lqjlaMK515We08vKinFNQUFB4YcpbGjOqRSV+23WdU0Gx1POwpU57Nw6vvUn4jXgqVrHW2b3TkjMJDYjLvmJJjNbGk3Zws6B+5wdrBj/8G0BkUjpzhtTFyEB5Hn4SilNQUFAoFEa08GT5iWC+2XqVDWObUKuFM2d23OHk1tt0e7d2od1HlaHm7s14QrKdQHRIMgBGxvo4V7PBr60bLtVtsKlgmrPb6GhgDMuP32FEc+0JbIUnozgFBQWFQsHUyID32lfjw7UX+OfCXbr6VXwwWrgZT8XK1s/VrkatITI4idCAWEKuxhFxKwGNWqJnIHDysqJhVy9cathQ3s0iRwb7YdIy1UxefwF3O1MmtXu54i0/D4pTUFBQKDR61XFh4eEgfvw3gPY1HbWjhf+CObkl/6MFKSVxd1MJvaZ1AuHX48hMV4MAexdz/Nq44lLDBqfK1jmidk9j6s5rBMeksmJEI0zyUf5lR3EKCgoKhYa+nuCzzjUYtOA4i48EMaplJWq3d+fIuqePFpLj0nPWBUICYklNyATA0sGEyvUdca1ui3M1a0zMjQpkz7mQeP48dJuBDd1oXMnuRbv3UqA4BQUFhUKlaWV72lQvz6y9N+lTzxXvFs6c3ZF7tJCRqiLsenzO4nBcRCoAJhaGOFezwbW6LS7VbXIOnD0PmVkaPlx7nvIWxkzuVL1Q+vYyoDgFBQWFQueTV6vTYdpBpu+6zpRu3jmjhX3LrxEdkkRkUCJSgoGRHhWr2FCjaUVca9hgV9EcUUg6RL/vvcn1e8n8NbwelsYvZ7zl50FnTkEIYQwcAMpl32etlPILod0O8A3QB1ADf0gpZwghrIClgFt2+V+klAt1ZZ+CgoLuqFzeggENXFl2/A5Dm3jg3cKZc7vucOVQOI4eltTt5IFrDRscPa3QL+TtobEpmRy/FcPsfTfp7l+RNtUdC7X9so4uRwoZQBspZbIQwhA4JIT4F6gBuALVpZQaIUT57PLjgCtSyi5CCAfgmhBimZQyU4c2Kigo6Ih3X6nKxrPhfL8tgAXD6jHg84bo6QmMTArvaychTcWlsAQuhCZwMSyeC6EJhMalAVDB0pjPu7zc8ZafB505BSmlBJKz3xpmXxIYAwyUUmqyy0XerwJYZI8kzIFYIEtX9ikoKOgWe/NyjG1diZ+2X+NoYMwLL/QmZ2RxOSyBizlOIIHb0Sk5+W62pvi5WjOkkTs+Llb4uVhjVk6ZIS8oQvvdraPGhdAHTgOVgd+llB8JIWKAqUAPIAoYL6W8IYSwADYD1QELoJ+UcmsebY4ERgK4ubnVDQ4O1pn9CgoKL0a6Sk3bX/djY2bI5nHN8h23IC1TzZW7iVwIjediaAIXwhIIjErm/tdVRStjfFys8HWxxtfFCh9nK6xNC7Yz6WVGCHFaSlkvrzydulEppRrwF0JYAxuEEN5o1xjSpZT1hBA9gb+A5kAH4BzQBqgE7BRCHJRSJj7S5jxgHkC9evV059EUFBReGGNDfT7sWI0JK8+x4WwYveq6PFYmI0tNwN0kLoQlcDFUOwV0IzIZtUb7721vXg4/Fyu6+FbE18UKb2crHCzKFXVXXhqKZGwlpYwXQuwFOgKhwPrsrA3A/cXk14EfsqedbgohbqMdNZwoChsVFBR0Qxffivx56Da/7LhG+1qO3IlNzXn6vxAaz7WIJFRqrQOwMTXE18WadjUd8XHWjgQcLcvlyFUo6B5d7j5yAFTZDsEEaAf8CGwEWgO3gZbA9ewqd4C2wEEhhCNQDbilK/sUFBSKBj09wWeda9J37lH8puwgewCAhbEBvi5WvNnMK2cKyMXGRHEAxYwuRwpOwOLsdQU9YLWUcosQ4hCwTAgxEe1C9FvZ5b8GFgkhLgIC+EhKGa1D+xQUFIqIBp62fNypOpFJGfhmrwW425oqsZFLIDpdaNY19erVk6dOnSpuMxQUFBRKFU9baFZExRUUFBQUclCcgoKCgoJCDopTUFBQUFDIQXEKCgoKCgo5KE5BQUFBQSEHxSkoKCgoKOSgOAUFBQUFhRwUp6CgoKCgkEOpPrwmhIgCnlcm1R4oiyemy2q/7lOW+1eW+3afstzH0tQ3dymlQ14ZpdopvAhCiFNPOtFXmimr/bpPWe5fWe7bfcpyH8tK35TpIwUFBQWFHBSnoKCgoKCQw8vsFOYVtwE6oqz26z5luX9luW/3Kct9LBN9e2nXFBQUFBQUHudlHikoKCgoKDyC4hQUFBQUFHIoNU5BCOEqhNgrhLgihLgshJiQnW4rhNgphLiR/dMmO10IIWYIIW4KIS4IIeo81NZ2IUS8EGLLM+45LLvdG0KIYQ+lfyuECBFCJJexfm0XQpzPtmNOdtS8stS/fUKIa0KIc9lX+bLQNyGExUN9OieEiBZCTHuRvpW0Pman98tu87IQ4sdS2rc8ywkh3s5uVwoh7F+0by+ElLJUXGjDe9bJfm2BNrZzTeAnYHJ2+mTgx+zXrwL/og3t2Qg4/lBbbYEuwJan3M8WbYxoW8Am+7VNdl6jbHuSy1i/LLN/CmAd0L+M9W8fUK8sfiYfKXcaaFGW+gjYoY3j7pBdbjHQtjT17WnlgNqABxAE2BfWZ/R5rlIzUpBS3pVSnsl+nQRcBZyBbmg/IGT/7J79uhuwRGo5BlgLIZyy6+8Gkp5xyw7ATillrJQyDtgJdMyuf0xKebcM9isxu4wBYAS88C6EktS/wqYk9k0IURUoDxx8we6RbVdJ6aMXcENKGZVdbhfQq5T17YnlpJRnpZRBL9KfwqLUOIWHEUJ4oPWsxwHHh76gIwDH7NfOQMhD1UKz0/LLi9YvMCWhX0KI/4BItB/ctQVo95mUhP4BC7OnWP4nhCi0qPElpG8A/YFVMvvxszAp5j7eBKoJITyEEAZov6hdC9iFJ1JEfSsVlDqnIIQwRzu18e5DT7YAZP8jlMo9tiWlX1LKDmiH1eWANoXVbgnp3yAppQ/QPPsaUhiNlpC+3ac/sKKwGy3uPmaPGsYAq9COgoIAdWG0Xdx9K2mUKqcghDBE+8dbJqVcn5187/4QLvtnZHZ6GLmfJFyy057UdsOHFuq6FrT+i1DS+iWlTAc2oR0uvzAlpX9Syvs/k4DlQIOy0rfs8n6AgZTy9At261E7SkQfpZT/SCkbSikbA9fQrgGUpr6VDp53MaKoL7SLO0uAaY+k/0zuRaGfsl93Jvei0IlH6rXi2Qtet9Euctlkv7Z9pExhLDSXiH4B5oBTdhkDtE9kb5eh/hmQvYAHGKKdGhtdFvr2UP4PwJTC+p8raX0Eymf/tAHOAVVLU9/yU44SsNBcbDd+jj9gM7TDuAvZH4hzaHcD2AG7gRtoF5/uf4AE8DsQCFzkoV0naIefUUAa2nnBDk+45xto5zJvAq8/lP5Tdj1N9s8vS3u/0M6bnsy24xIwE+1TZ5n4uwFmaHflXAAuA9MB/bLQt4fybgHVy/D/3QrgSvZVGDvjiqNveZYDxme/zwLCgQWF+XcsyKXIXCgoKCgo5FCq1hQUFBQUFHSL4hQUFBQUFHJQnIKCgoKCQg6KU1BQUFBQyEFxCgoKCgoKOShOQUGhAAgh1NmHkS4LraLse0KIp/4fZUszDCwqGxUUXgTFKSgoFIw0KaW/lLIW0A7oBHzxjDoegOIUFEoFyjkFBYUCIIRIllKaP/TeC+2hP3vAHfgb7UE50J4IPyKE+H97d8wSRxSFYfj9wCDBwsreRkMak8IfsJWVpbYpAyns0qdJbRGMxC5Wqez1BwhWov6BlCGkCYhgkd2T4s5OIERhDeyG8D7dzNyBudXHvZc55wx4Svs79xB4R/v7eECrMfW+qg6mNgnpHoaCNIHfQ6G79x14QqssO6qq2yQrwKeqWk8yAF5X1WY3/iWtZMPbJPPAKbBdVZ+nOBXpj+Zm/QHSf+QRsJfkOa2C5+od4zaAtSRb3fUisEJbSUgzZShIf6HbPhrSKmm+Ab4Cz2jndbd3vQbsVNXJVD5SmoAHzdIDJVkCPgB71fZhF4EvVTWi9WoY97i+prV7HDsBXnVlm0mymmQB6R/gSkGazOMkF7Stoh+0g+Xd7tk+cJTkBXAM3HT3r4BhkkvgI61C6zJw3nV/+8avlo/STHnQLEnquX0kSeoZCpKknqEgSeoZCpKknqEgSeoZCpKknqEgSer9BLP+iRWjqXu3AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(X_test, y_test, label='Actual level')\n",
    "plt.plot(X_test, flaml_y_pred, label='FLAML forecast')\n",
    "plt.plot(X_test, prophet_y_pred, label='Prophet forecast')\n",
    "plt.plot(X_test, autoarima_y_pred, label='AutoArima forecast')\n",
    "plt.plot(X_test, autosarima_y_pred, label='AutoSarima forecast')\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('CO2 Levels')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.x",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
